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Resistance to High Stakes Testing Spreads 
A national resolution to limit standardized testing is gathering support. 

By: Bob Schaeffer 

District Administration, September 2012 

Resistance to High Stakes Testing SpreadsMembers of the American Federation of Teachers react to comments made at 
their annual conference this past summer in which AFT President Randi Weingarten stated that high-stakes testing 
should be used to inform, not impede, instruction. AFT unanimously passed a resolution that says the focus on 
standardized tests has undermined the nation’s education system 

A rising tide of protest is sweeping across the nation as growing numbers of parents, teachers, administrators and 
academics take action against high-stakes testing. Instead of test-and-punish policies, which have failed to improve 
academic performance or equity, the movement is pressing for broader forms of assessment. From Texas to New York 
and Florida to Washington, reform activists are pressing to reduce the number of standardized exams. They also seek to 
scale back the consequences attached to test scores and use multiple measures to evaluate students, educators, 
schools and districts. 

The nation’s second-largest teachers union also took a stand recently against high-stakes testing, passing a resolution in 
July at its annual convention in Detroit that says the focus on standardized tests has undermined the United States’ 
education system. The American Federation of Teachers approved the resolution unanimously, stating that testing 
should be used to inform and not to impede classroom instruction. “It’s time to restore balance in our schools so that 
teaching and learning, not testing, are at the center of education,” stated AFT President Randi Weingarten “Test-driven 
education policies continue to force educators to sacrifice time needed to help students learn to critically analyze content 
and, instead, focus on teaching to the test. And students lose out on rich learning experiences when districts cut art, 
music, sports, social studies, science and other subjects to focus strictly on math and reading tests.” 

Of course, opposition to high-stakes testing is not new In the early years of NCLB and state-mandated exams, scattered 
boycotts of those tests took place in communities such as Scarsdale, N.Y., and Cambridge, Mass. What is very different 
in 2012 is the breadth and depth of the protests. Never before have large numbers of school board members, 
administrators, principals and parents stood up to challenge testing policies. 

The current movement gained significant momentum, oddly enough, in Texas, the state where many high-stakes testing 
practices began. The catalyst was a January 2012 statement by Robert Scott, the former state superintendent of schools 
who left the office in July, in which he called the belief that standardized testing is the “end-all, be-all” of education a 
“perversion.” Scott also labeled “the assessment and accountability regime” not only “a cottage industry but a military- 
industrial complex.” Almost immediately, local school boards began endorsing resolutions charging that overreliance on 
high-stakes exams is “strangling” classrooms. So far, nearly 550 local school boards in Texas have signed on, including 
those in big cities such as Dallas, Houston and San Antonio as well as those in hundreds of smaller communities. All 
told, these school boards represent districts that are respons ble for educating 3.3 million Texas students, or more than 
half of the state’s public school enrollment. 

Meanwhile, in New York state, more than 1,400 principals from urban, suburban and rural schools signed a letter 
protesting the state's new test-centric teacher-evaluation policy. Their statement concludes with a reminder that a 2011 
report by the National Research Council found that the past decade’s emphasis on testing had produced little learning 
progress A series of errors in writing, administering and scoring this year’s New York State Regents exams accelerated 
the movement. Most notorious was “Pineapplegate” in which several questions on the exam about a poorly written 
reading passage titled “The Hare and the Pineapple" had no coherent answers. 

A National Resolution Is Born 

Responding to the enthusiastic embrace of the Texas resolution and educators’ statements, the National Center for Fair 
and Open Testing (FairTest) spearheaded an effort this past spring to craft a statement that would appeal to a broader 
audience. The result, the National Resolution on High-Stakes Testing, gained initial sponsorship from a dozen other 
education, civil rights and religious groups, including the NAACP Legal Defense Fund and Educational Fund, the United 
Church of Christ’s Justice and Witness Ministries, Parents Across America and the National Education Association 
Many local groups, including Time Out from Testing in New York City and Parents United for Respons ble Education in 
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The resolution urges state officials to "reexamine school accountability.” It calls for a system “which does not require 
extensive standardized testing, more accurately reflects the broad range of student learning, and is used to support 
students and improve schools.” It also asks Congress and the Obama administration to overhaul NCLB. At the federal 
level, the resolution's goal is “to reduce the testing mandates, promote multiple forms of evidence of student learning and 
school quality in accountability, and not mandate any fixed role for the use of student test scores in evaluating 
educators.” 

As of mid-June, more than 10,000 individuals from all 50 states and 350 organizations, including the National Council of 
Teachers of English and parents groups in Tucson, Baton Rouge, Minneapolis and Charlotte, had signed the resolution. 

The National Resolution, in turn, has stimulated activists in several regions to press for their own versions. In Florida, 
another state where high-stakes testing had long dominated the education reform debate, a dozen countywide school 
boards signed on within a three-week period. Endorsers included Broward County Public Schools, the nation’s sixth- 
largest district, and the School District of Palm Beach County. With grassroots resolutions sweeping the state, the Florida 
School Board Association took up a version at its annual conference last spring. Despite a condescending lecture from 
the state education commissioner warning delegates against passage, they voted to adopt it overwhelmingly, even 
though they are still legally required to administer the controversial FCAT exam. 

And, three school boards in the Tulsa, Okla., area, as well as boards in Ohio and Virginia, endorsed the resolution. The 
national Parent Teacher Association issued a statement noting that its policy positions were consistent with the 
resolution. 

Boycotts Pepper the Nation 

Diane Ravitch, research professor of education at New York University, speaking at the NCTM regional conference this 
past summer, discussing the impact of school reforms.Aside from extensive media coverage about the movement, New 
York parents this past spring organized a boycott of a "field test” designed to try out potential questions for future exams. 

Parents at more than five dozen schools refused to allow their children to take the trial exams. 

Boycotts also spread in other states, including Colorado and California. In Snohomish (Wash.) School District, 550 
parents stopped their children from taking the Measurements of Student Progress test, the state’s exam for third- through 
eighth graders, and are working to promote test refusal in other communities. 

What’s behind the surge of criticism of high-stakes testing and support for assessment alternatives? Several forces are at 
work. First, and most important, is the widespread recognition that test-driven education “reform,” embodied by NCLB 
and state graduation tests, has failed. Multiple statistical studies, such as FairTest's report “NCLB’s Lost Decade for 
Educational Progress,” have shown that federally mandated testing did not increase average academic performance or 
narrow achievement gaps significantly. In fact, U.S. students made greater gains on the National Assessment of 
Academic Progress (NAEP) before NCLB became law. Reports by independent experts, including the National Research 
Council, have found little evidence that proves high-stakes testing has improved academic performance among students. 

Second, parents and educators saw that test-driven schooling damaged educational quality and equity by narrowing 
curriculum and focusing on the limited skills that standardized tests measure. These negative effects fell most heavily on 
classrooms serving low-income and minority children. 

Third, a series of errors in test construction, administration, scoring and reporting damaged the industry’s credibility. The 
public now understands that the tools politicians mandate to enforce educational “accountability” are produced by 
unaccountable companies focused on generating profits, not helping children learn. 

Finally, test-cheating scandals have undermined confidence in policies that rely on standardized exams. Investigations in 
Atlanta, Baltimore, Denver, El Paso, Flouston, Indianapolis and New York City have shown that many highly promoted 
gains resulted from score manipulation. Improper behaviors range from erasing wrong answers to barring certain 
students from school on testing days. According to FairTest, cheating cases have been confirmed in 36 states and the 
District of Columbia over the past four years. 

The Alternatives 

Robert Scott, former Texas education commissioner, publicly stated that the idea that high-stakes testing was the end-all, 
be-all was a "perversion,” creating a greater push for the movement. Overwhelmed by the evidence, defenders of high- 
stakes testing typically fall back on one last-ditch argument: “So what’s the alternative?” 

Flowever, many concrete proposals made by groups like FairTest, the Forum on Educational Accountability, and the 
Broader, Bolder Approach to Education have demonstrated that better methods for evaluating student progress already 
exist. Assessment based on student performance on real learning tasks is more useful and accurate for measuring 
achievement than any multiple-choice test. 

Trained teams of educators can also be used to rate academic performance. Such a process is already used to grade the 
non-multiple-choice portions of Advanced Placement exams. Studies have shown that with training, the level of 
agreement on grading among judges is high. As with multiple-choice tests, safeguards are necessary to ensure that 
race, class, gender, linguistic or other cultural biases do not affect evaluation. 

The United States is the only economically advanced nation that relies heavily on multiple-choice tests. Other nations, 
such as Finland, primarily use performance-based assessments. Their students are evaluated based on real academic 
work such as essays, projects and activities. Ironically, because these nations do not focus on teaching to multiple- 
choice exams, they even score higher than U.S. students do on those tests, as several researchers, including Finnish 
policy analyst Pasi Sahlberg, have demonstrated. 

Using Campaign to Close Gap 
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the Broad, Gates and Walton foundations—on the other. Public opinion polls consistently show support for cutting back 
on high-stakes testing. But policy makers and their close supporters continue to defend the status quo, even though the 
evidence reveals that it has not succeeded. 

This past summer and this fall, the assessment reform groups who initiated the National Resolution are using the election 
campaign season to close that gap. The groups plan to press all candidates for Congress, local offices and even as high 
as the president to take public positions against test misuse and overuse. 

In St. Petersburg, Fla., voters convinced seven of eight contenders for the Pinellas County School Board to support a 
statement opposing high-stakes standardized exams. By “bird-dogging” candidate forums, publishing letters-to-the-editor 
in local media, commenting on political blogs, and asking pointed questions, advocates expect to deliver a clear 
message to those who ultimately make assessment policy. 

Leaders of the testing reform movement are realists. They know that a winning campaign requires more than one 
resolution or a single electoral cycle. They are confident, however, that the increasing power of public opinion will 
ultimately lead policy makers to roll back excessive high-stakes standardized exam mandates and finally adopt better 
forms of assessment. 

Last May, California Senate President Pro Tern Darrell Steinberg speaks with Sen. Elaine Alquist after his school testing 
bill, SB1458, was approved by the Senate. The bill, which makes schools less reliant on student testing, is part of a 
larger education package under consideration. If passed in the Assembly, it would need governor approval. Last May, 
California Senate President Pro Tern Darrell Steinberg speaks with Sen. Elaine Alquist after his school testing bill, 
SB1458, was approved by the Senate. The bill, which makes schools less reliant on student testing, is part of a larger 
education package under consideration. 

Bob Schaeffer is the public education director of the National Center for Fair and Open Testing. Colleagues Monty Neill 
and Lisa Guisbond contributed to this article. 


JA2885 


http://www fairtest org/resistance-high-stakes-testing-spreads[l/20/2016 1:29:53 PM] 


Case l:14-cv-00857-TSC Document 67-16 Filed 01/21/16 Page 1 of 3 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 33 of 441 


EXHIBIT 14 


JA2886 


California Intellectual Property Laws 

Case l:14-cv-00857-TSC Document 67-16 Filed 01/21/16 Page 2 of 3 
USCA Case #17-7035 Document #1715850 S| g n l Pt1§Eh W31/20^8 er ( f^ge s 94 c< of Feedback 


^0 LexisNexis* Store 


Advanced 
_J Search. 


NEED HELP? 

latott 877 394 8826 
pport: 800 833 9844 


shop by: Practice Area T Jurisdiction T Products ’ Law School 


Law Enforcement 


CART (0 Items) Q 


Home > Shop by Practice Area > California Intellectual Property Laws, 2015 Edition 



California Intellectual Property Laws, 
2015 Edition 

Publisher: Matthew Bender 

Includes complete and fully updated provisions of all relevant 
California and federal intellectual property statutes with expert 
commentary. 




International Oi 


Recommended Products 



information on this title 


may be available to cust 
that subscribe to both a 




The most cited patent l< 
treatise today, providin 
definitive analysis of all 
involving patent law do< 
rules and case law. 


Print Book 

ISBN:9781632810083 

Quantity: £1 [Tj [±] 

$146.00 

“ 

eBook: epub 

ISBN. 9781632810090 

Quantity: d Q [±l 

$146.00 


eBook: mobi 

ISBN:9781632810090 

Quantity: id Q [±] 

$146.00 





Print 

Description 

Table of Contents Volume Pricing 

Reviews 



■ 


lexis Practice Advisor® 
Intellectual Property & 
Technology is a 


critical to IP practitioners lik 


practical guidance content 
written by leading 




Moores federal Practice 


Get complete and fully updated text provisions for all relevant California and intellectual property statutes 
along with expert commentary by Leech Tishman Fuscaldo & Lampl with California Intellectual Property 
Laws. Includes an Introduction that provides an overview of intellectual property law and Finding Aids that 
consist of topical references to Federal and California statutes. 

Key California statutes are accompanied by consultant comments and annotations. Consultant comments 
reflect the observations, opinions, and practical experience of the Editorial Consultant. Annotations are also 
included to help direct the practitioner to key cases and secondary sources. 

The eBook version of this title features links to Lexis Advance for further legal research options. 


Moores Federal Practice 

Complex Litigation, Fourth. 
Issued in 2004 under the 
auspices of the Federal Judicial 
Center, Washington, D.C., the 
Fourth Edition of the Manual is 

federal litigators. 



and the undisputed leading 
authority for in-depth, 

copyright law. 


SEEING DOUBLE 

Save 20%* on two or more items! 

'Limited-lime o*er. See complete details. 



SAVE NOW 


JA2887 


http://wv 


ctdetarl jsp?pageName=relatedProducts&prodId=10291[l/20/2016 8:55:23 PM] 


BE3E0Q 




















California Intellectual Property Laws 

Case l:14-cv-00857-TSC Document 67-16 Filed 01/21/16 Page 3 of 3 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 35 of 441 


Contact a Sales Person: 1-877- 


□ n 


Copyright © 2016 LexisNexis. All rights reserved. 


JA2888 


http://wv 


ctdetail jsp?pageName=relatedProducts&prodId=10291[l/20/2016 8:55:23 PM] 






Case l:14-cv-00857-TSC Document 67-17 Filed 01/21/16 Page lot 4 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 36 of 441 


EXHIBIT 15 


JA2889 


Code of Federal Regulations, Title 38, Pensions. Bonuses, and Veterans'Relief, Pt 0-17, Revised as of July 1,2015: Office of the Federal Register (U S ): 9780160929229: Amazon com: Books 

Case l:14-cv-00857-TSC Document 67-17 Filed 01/21/16 Page 2 of 4 

_ I |c;r!A ra<;p #17-703R Dnn impnt #1 71 _ Filprl - 01 /31 /9D1 8 Panp 37 nf AA 1 


hy prime New Year, New You Sponsored by Inturt 

Shop by Hello. Sign in Tty Your 0 

Department - Your Amazon.com Today's Deals Gilt Cards Sell Help Your Account - Prime - Lists - 


Books Advanced Search New Releases Best Sellers The New York Times® Best Sellers Children's Books 



Code of Federal Regulations, 
Title 38, Pensions, Bonuses, 
and Veterans' Relief, Pt. 0-17, 
Revised as of July 1, 2015 

Paperback - November 20,2015 

by Office of the Federal Register (U S.) (Compiler) 
Be the first to review this item 

► See all formats and editions 

Paperback 

$66.00 


8 Used from $61.70 
12 New from $56.52 



See this image 


Best 
Books 
of 2015 


Looking for something great to read? 
Browse our editors' picks for 2015"s Best 
Books of the Year in fiction, nonfiction, 
mysteries, children's books, and much 


Textbooks Textbool 


New customer? Start here. 
Share ETH if © 

' * Buy New $66.00 

Qty [TJ 

FREE Shipping. 

Temporarily out of stock. 

Order now and we'll deliver when 
available. Details ▼ 

Ships from and sold by Amazon.com. 
Gift-wrap available. 

□ Yes, I want FREE Two-Day 
Shipping with Amazon Prime 


Sign in 


Turn on 1-Click ordering for this browser 


Ship to: 

SANTACLARA, CA 95050 - 


Customers Who Viewed This Item Also Viewed 


Code of Federal 
Regulations, Title 38 - 
Parts 18-end Pensions, 
Bonuses & Veterans Relief 
National Archives and... 
Paperback 


\ CFR 



Code of Federal 
Regulations Title 38, 
Pensions, Bonuses and 
Veterans' Relief, Parts 0- 
National Archives and... 
Paperback 
$66.00 V Prime 



Code of Federal 
Regulations, Title 38, 
Pensions, Bonuses, and 
Office of the Federal... 

$66.00 uptime 


o 


Buy Used 


$61.70 


Have one to sell? 


Seller 


Code of Federal 
Regulations, Title 38, 


JA2890 


)/0160929229/ref=sr_l_15?s=books&ie=UTF8&qi<l=1453340653&sr=l-15[l/20/2016 8:58:26 PM] 



























Code of Federal Regulations, Title 38, Pensions. Bonuses, and Veterans' Relief, Pt 0-17, Revised as of July 1,2015: Office of the Federal Register (U S ): 9780160929229: Amazon com: Books 

Case l:14-cv-00857-TSC Document 67-17 Filed 01/21/16 Page 3 of 4 
F^9©A B ©ffise#17-7035 Document #1715850 Filed: 01/31/2018 Page 38 of 441 


Veterans' Relief, Pt. 18- 
Office of the Federal... 
Paperback 


Customers Viewing This Page May Be Interested In These Sponsored Links (what'sthis?) 

1. Veterans Codes - Learn About Veteran Laws. Find Lawyers for Free at FindLaw. Iawyers.findlaw.com/Veteran 


Ad feedback 


Special Offers and Product Promotions 


• Your cost could be $36.00 instead of $66,001 Get the Amazon.com Rewards Visa card and you’ll automatically get $30.00 off instantly as a gift card. 
Apply now. 


Product Details 

Paperback: 872 pages 

Publisher: Office of the Federal Register; Revised edition (November 20,2015) 
Language: English 
ISBN-10: 0160929229 
ISBN-13: 978-0160929229 
Product Dimensions: 5.8 x 1.5 x 9.1 inches 
Shipping Weight: 12.6 ounces (View shipping rates and policies) 

Average Customer Review: Be the first to review this item 
Amazon Best Sellers Rank: #1,333,756 in Books (See Top 100 in Books) 
#996 in Books > History > Military > United States > Veterans 
#1258 in Books > Law > Tax Law 
#3361 in Books > Law > Administrative Law 



Tell the Publisher 

I'd like to read this book on Kindle 

Don’t have a Kindle? Get your 
Kindle here, or download a FREE 
Kindle Reading App. 


Would you like to update product info or give feedback on images? 


Customer Reviews 

There are no customer reviews yet. 

5 star Share your thoughts with other customers 

4 star 

3 star Write a customer review 

2 star 
1 star 


amazon Prime 

There's more to 
Prime than FREE 
Two-Day Shipping 

| Try it free for 30 days 


Q 


|P 

Ad feedback Q 


P 

JA2891 


http://www amazon com/Fedeial-Regulations-Pensions-Bonuses-Veterans/dp/0160929229/ref=sr_l_15?s=books&ie=UTF8&qid=1453340653&sr=l-15[l/20/2016 8:58:26 PM] 






egulations, Title 38, Pensions, Bonuses, and Veterans' Relief, Pt 0-17, Revised as of July 1,2015: Office of the Federal Register (US): 9780160929229: Amazon com: Books 

Case l:14-cv-00857-TSC Document 67-17 Filed 01/21/16 Page 4 of 4 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 39 of 441 


Back to top 


Get to Know Us 
Careers 
About Amazon 
Investor Relations 
Amazon Devices 


Make Money with Us 

Sell on Amazon 
Sell Your Services on Amazon 
Sell on Amazon Business 
Sell Your Apps on Amazon 
Become an Affiliate 
Advertise Your Products 
Self-Publish with Us 
Become an Amazon Vendor 


Amazon Payment Products 
Amazon.com Rewards Visa Card 
Amazon.com Store Card 
Amazon.com Corporate Credit Line 
Shop with Points 
Credit Card Marketplace 
Amazon Currency Converter 


Let Us Help You 
Your Account 
Your Orders 

Shipping Rates & Policies 
Amazon Prime 
Returns & Replacements 
Manage Your Content and Devices 
Help 


amazon.com 


5a- 


s_ 


S£a— 

ssr 

SSfe. 

Si£T 

—L. 

SET’ 



aag 

fe. 

SSL 

fndteP^i? Publishing 

sl; 





&L 

3F" 

agMSST" 

was, 

~ 

sa¬ 

Sasi 

S:IL 

SI 

SSL 

as 0 ”'- 


fe. 

sss 


JA2892 


ans/dp/0160929229/ref=sr_l_15?s=books&ie=UTF8&qid=1453340653&sr=l-15[l/20/2016 8:58:26 PM] 



Case l:14-cv-00857-TSC Document 67-18 Filed 01/21/16 Page lot 4 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 40 of 441 


EXHIBIT 16 


JA2893 


BARNES&NOBLE 


SPEND $25, 

Get 

FREE SHIPPING 


Sign In My Account * 


Save $5 on Two Business Books 


Membership Gift Cards Stores & Events Help 


\nnv RaaVc rirtnic 

You are looking at 


Barnes & Noble Classics x 


Barnes & Noble Classics 


Fiction 

Fiction & Literature 
Classics 

American Fiction & 
Literature Classics 


Refine by 


Prices 


Under $5 


$5-$10 


$10-$25 


$25 - $50 


Formats 


Paperback 


NOOK Book 


Hardcover 



ADVERTIS NG 


D> 


Showing l - 20 of 64 results 



Essential Tales and 
Poems of... 
by Edgar Allan Poe 

***4 

Paperback $7.88 
NOOK Book $4.99 



Moby-Dick (Barnes 
& Noble... 
by Herman Melville 

Paperback $8.60 
NOOK Book $3.99 



The Scarlet Letter 
(Barnes &... 
by Nathaniel 
Hawthorne 

Paperback $6.26 
NOOK Book $3.99 



Adventures of 
Huckleberry Finn... 
by Mark Twain 

4kik4k4 

Paperback $6.25 
NOOK Book $3.99 



Narrative of the Life 
of... 

by Frederick 
Douglass 

***4 

Paperback $6.25 
NOOK Book $3.99 



Little Women 
(Barnes & Noble... 
by Louisa May 
Alcott 

*** 

Paperback $7.15 
NOOK Book $3.99 



Adventures of Tom 
Sawyer... 
by Mark Twain 

+ ++4 

Paperback $6.25 
NOOK Book $3.99 



The Beautiful and 
Damned... 
by F. Scott 
Fitzgerald 

★ ★★ 

Paperback $8.05 
NOOK Book $4.49 



The Jungle (Barnes 
& Noble... 
by Upton Sinclair 
ikikiki 

Paperback $8.95 
NOOK Book $3.99 



Uncle Tom's Cabin 
(Barnes &... 
by Harriet Beecher 
Stowe 
ikikiki 
Paperback $7.15 
NOOK Book $3.99 


JA2894 


rican-fiction-literature-classics/_/N-rqvZllhj[l/20/2016 9:00:18 PM] 



































The Legend of 
Sleepy Hollow... 
by Washington 
Irving 

Paperback $8.05 
NOOK Book $3.49 


Call of the Wild and 
White... 
by Jack London 

Paperback $6.25 
NOOK Book $3.99 


The Collected Poems 
of Emily... 
by Emily Dickinson 

Paperback $7.15 
NOOK Book $4.49 


This Side of Paradise 
(Barnes ... 
by F. Scott 
Fitzgerald 

Paperback $8.95 


The Last of the 
Mohicans... 
by James Fenimore 
Cooper 

★ 

Paperback $8.95 



Age of Innocence 
(Barnes &... 
by Edith Wharton 



My Antonia (Barnes 
& Noble... 
by Willa Cather 



Incidents in the Life 
of a... 

by Harriet Jacobs 

**** 



House of Mirth 
(Barnes & Noble... 
by Edith Wharton 



Leaves of Grass: 

First and... 
by Walt Whitman 


Paperback $8.05 
NOOK Book $3.99 


Paperback $6.25 
NOOK Book $3.99 


Paperback $8.05 
NOOK Book $3.49 


Paperback $7.15 
NOOK Book $4.49 


2 I 3 | > 


ADVERTIS NG 


wayfair up to 70 % off 

Shop Now ► Everything Home 





Visit Your Local Store 
Find AuthfivandsStorytime 
Find a Store > 



Gift Cards 
Always the Perfect Gift 
Shop Now > 



Member 

BARNES !t NOBLE 

BN.COM 


B&N Membership 
Don't Miss Out on Savings 
Learn More > 


JA2895 


http://www bamesandnoble com/b/bames-noble-classics/fiction- 


rican-fiction-literature-classics/_/N-rqvZllhj[l/20/2016 9:00:18 PM] 






























Barnes & Noble Classics, American Fiction & Literature Classics, Fiction & Literature Classics | Bames & Noble 

Case l:14-cv-00857-TSC Document 67-18 Filed 01/21/16 Page 4 of 4 
B&N senicesyscA Case #17-7©86"g & ©foment #1715889“ Hel P Filed: 01/31/2018 storf PSge‘43 of 441 


Advertise 


About Free Shipping 


Customer Service 


Author Events, FREE Wi-Fi, and more 


Publisher & Author Guidelines 
Self-Publish with NOOK Press 
Bulk Order Discounts 
B&N Membership 
B&N MasterCard 
B&N Kids' Club 
B&N Educators 
B&N Bookfairs 
Careers at B&N 


About Shipping 
Shipping Rates 

NOOK 

NOOK Tablets & eReaders 
NOOK Mobile Apps 
Special Offers 
NOOK on Facebook 
NOOK on Twitter 


Order Status 
Easy Returns 
Product Recalls 


About Us 
About B&N 
Investor Relations 
Bames & Noble, Inc. 


y Store Locator 


Be in the Know 

Sign up for savings, news, updates 


Email Address 


Submit 


f * p 


Terms of Use Copyright & Trademark 


Privacy Policy 


Sitemap Accessibility 


© 1997-2016 Bames & Noble Booksellers, Inc. 
122 Fifth Avenue, New York, NY 10011 


JA2896 


rican-fiction-literature-classics/_/N-rqvZllhj[l/20/2016 9:00:18 PM] 








USCA Case #17-7035 Document #1715850 


Filed: 01/31/2018 Page 44 of 441 


MATERIAL UNDER SEAL DELETED 


JA2897-JA3066 


Case l:14-cv-00857-TSC Document 69-5 Filed 01/21/16 Page 1 of 11 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 45 of 441 

UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN EDUCATIONAL RESEARCH 
ASSOCIATION, INC., AMERICAN 
PSYCHOLOGICAL ASSOCIATION, INC., and 
NATIONAL COUNCIL ON MEASUREMENT IN 
EDUCATION, INC., 

Case No. l:14-CV-00857-TSC-DAR 

DECLARATION OF CARL 

MALAMUD IN SUPPORT OF 
DEFENDANT-COUNTERCLAIMANT 
PUBLIC.RESOURCE.ORG’S MOTION 

Plaintiffs, 

FOR SUMMARY JUDGMENT AND 
OPPOSITION TO PLAINTIFFS’ 

V. 

MOTION FOR SUMMARY 

PUBLIC.RESOURCE.ORG, 

JUDGMENT AND PERMANENT 
INJUNCTION 

Defendant. 

Action Filed: May 23, 2014 


JA3067 




Case l:14-cv-00857-TSC Document 69-5 Filed 01/21/16 Page 2 of 11 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 46 of 441 

I, Carl Malamud, declare as follows: 

1. Iam over the age of 18 years and am fully competent to testify to the 
matters stated in this declaration. 

2. This declaration is based on my personal knowledge. If called to do so, I 
would and could testify to the matters stated herein. 

3. I am the President and sole employee of Public.Resource.Org, Inc. 

(“Public Resource”), which is a 501(c)(3) non-profit corporation headquartered in 
Sebastopol, California. I have worked at Public Resource since I founded the 
organization in 2007. It is my only source of employment. 

4. Public Resource’s core mission is to make the law and other government 
materials more widely available so that people, businesses, and organizations can easily 
read and discuss our laws and the operations of government. Attached to Public 
Resource’s Consolidated Index of Exhibits as Exhibit 1 is a true and correct copy of 
Public Resource’s Articles of Incorporation from our website at 
https://public.resource.org/public.resource.articles.html . 

5. That mission grows out of my longtime professional commitment to 
improving public access to essential documents that shape our fundamental activities. In 
1991,1 convinced the Secretary-General of the International Telecommunication Union 
that the Blue Book, the specification for how telephone networks operate, should be 
freely available on the Internet. Working with Dr. Michael Schwartz, I transformed and 
posted the Blue Book into formats compatible with modem publication technologies and 
made it available on the Internet. The service was extremely popular, and the ITU today 
makes all of its standards documents freely available on the Internet. I wrote a book about 
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this experience called “Exploring the Internet” (Prentice Hall, 1993).That book can be 
viewed and read at http://museum.media.org/eti/Exploring the Intemet.pdf . 

6. I was privileged to be able to participate in the Internet Engineering Task 
Force, the standards body that has developed most of the standards that specify the 
functioning of the Internet, during the early 1990s, a period of very rapid development, 
both in the functionality of the Internet and its scope. 

7. In 1993, when the Internet was beginning to grow explosively, I created 
the first radio station on the Internet, operating as a nonprofit corporation called the 
Internet Multicasting Service. In addition to transmitting audio and video programming, 
the service also provided the first high-speed Internet link into the White House, using a 
temporary infrared connection from our studios in the National Press Building. The radio 
service, which I dubbed “Internet Talk Radio,” became a member of the Public Radio 
Satellite System, received accreditation from the U.S. House and Senate Radio & 
Television Correspondents Galleries, sent out live audio from the floors of the House and 
Senate, streamed all National Press Club luncheons, and transmitted original 
programming. Many of those programs can still be listened to at http://museum. 
media.org/radio /. 

8. At the Internet Multicasting Service, I also put a number of important 
government databases online, including the Securities and Exchange Commission 
EDGAR database and the U.S. Patent database. When the SEC took the EDGAR service 
over from me, I loaned it computers and donated all of our source code so they could be 
up and running quickly. The SEC ran the system on our software for several years. On 
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October 10, 1995, the Hon. Arthur Levitt, Chairman of the SEC, wrote to me thanking us 
for our efforts and calling the project an “extraordinary achievement.” 

9. After I started Public Resource in 2007, one of our first efforts was to 
place online the historical opinions of the U.S. Courts of Appeals, material that was not 
previously available on the Internet. Public Resource also converted all of the opinions in 
the first 40 volumes of the Federal Reporter as well as the Federal Cases into Hypertext 
Markup Language (HTML) and placed those online. These materials are now used by 
numerous websites that provide access to legal materials. 

10. Public Resource maintains an archive of laws and other government 
authored materials on several domains under the public.resource.org website. 

11. Public Resource has helped increase access to many other court 
documents. We scanned approximately 3 million pages of briefs submitted to the U.S. 
Court of Appeals for the Ninth Circuit dating back to the creation of that court and have 
placed those materials online. The materials may be downloaded from 
https://law.resource.org/pub/us/case/ca9/ . 

12. Public Resource has conducted a number of other projects that have 
resulted in more government information being placed online. Using volunteers in 
Washington D.C. with the cooperation of the Archivist of the United States, we put 
approximately 6,000 government videos on YouTube and the Internet Archive for people 
to use with no restriction, a service we call FedFlix. It has had over 60 million views. The 
videos may be viewed at https://www.youtube.com/user/PublicResourceOrg and 
https://archive.org/details/FedFlix . 
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13. Public Resource also placed over eight million Form 990 exempt non¬ 
profit organization returns obtained from the IRS on the Internet. As part of that posting, 
we conducted an intensive privacy audit which led to fundamental changes in how the 
IRS deals with privacy violations. Through a Freedom of Information Act request and 
litigation, we obtained release of high-quality versions of Form 990 filings, which the 
IRS had refused to make available. The court decision in that case 

( Public.Resource.Org v. United States Internal Revenue Service, No. 3:13-cv-02789- 
WHO, ECF No. 62 (N.D. Cal. January 29, 2015)) led to a recent announcement by the 
IRS that all e-file returns will be made available in bulk in 2016.1 am pleased to be 
working with the IRS as a member of the test group for this service. 

14. In 2007,1 wrote a report addressed to Speaker of the House Nancy Pelosi 
suggesting that video from Congressional hearings should be more broadly available on 
the Internet. On January 5, 2011, Speaker John Boehner and Representative Darrell Issa 
wrote to me asking me to assist them in carrying out that task. In a little over a year, 

Public Resource was able to put over 14,000 hours of video from hearings on the 
Internet, to assist the House Committee on Oversight and Government Reform in posting 
a full archive of their committee video and, for the first time ever for congressional 
hearings, to provide closed-captioning of those videos based on the official transcripts. 

The letter from Speaker Boehner may be found at 
https://law.resource.org/rfcs/gov.house.20110105 .pdf . 

15. Also in 2008,1 examined the issue of availability of state-mandated safety 
codes, such as building, electric, plumbing, and fire codes. At the time, none of those 
documents were available freely on the Internet. I made a detailed survey of state 
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regulations and statutes, looking for direct and specific incorporation of particular model 
codes. Over the next few years, Public Resource posted many of the incorporated state 
safety codes for U.S. states. 

16. Public Resource’s process of posting these codes has been deliberate and 
careful and has grown in sophistication over time. First, we purchased paper copies of 
codes that are incorporated into law. Then, we scanned the documents, applied metadata 
and optical character recognition (OCR) to the PDF files, and placed a cover sheet on 
each document explaining that this was a posting of the law of a specific jurisdiction. 

17. Over time, we also began converting some of these standards into modern 
HTML format, including setting the tables, converting formulas to Mathematics Markup 
Language (MathML), and converting graphics to the Scalable Vector Graphics (SVG) 
format. Coding formulas in MathML makes them significantly more accessible to people 
who are visually impaired. Converting the graphics to SVG means they can be resized 
smoothly, and can be incorporated into graphic editing programs and word processing 
programs. Converting the documents into standard HTML means the documents can be 
more readily used on different platforms, such as tablets and smartphones. 

18. In late 2008,1 was asked by the Obama-Biden Transition Project to 
consult on the subject of how the Official Journals of Gover nm ent could be made more 
readily available. Many of my recommendations were adopted, including removing the 
subscription fee from bulk access to the Federal Register. That led to a dramatic 
transformation of the Federal Register, which is now based on open source software that 
was developed by three volunteers in California and then adopted by the government. 

That system can be viewed at https://federalregister.gov/ . A copy of my memorandum to 
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the Obama Transition Project may be viewed at 
https://public.resource.org/change.gov/reboot.register.pdf . 

19. In 2011,1 began to look seriously at the federal use of standards 
incorporated by reference into the Code of Federal Regulations. I was participating at the 
time as an appointed member of the Administrative Conference of the United States, and 
I carefully read materials such as the legislative history of the mechanism of 
incorporation by reference, the Code of Federal Regulations provisions for incorporation 
by reference, and cases such as the Veeck decision. 

20. In 2012,1 began a new initiative to make standards incorporated by 
reference into federal law available on the Internet. I examined the Code of Federal 
Regulations carefully and selected 73 standards that spanned a variety of agencies. I 
purchased physical copies of each of these standards. I created 25 paper replicas of each 
of these standards, and placed a cover sheet on each one indicating which section of the 
CFR incorporated the document. 

21. To accompany the 73 standards, I also created a detailed cover memo, 
titled “Notice of Incorporation,” which included letters addressed to seven senior 
government officials. The memo included a request for comments from each of the ten 
standards development organizations (SDOs) named in the document by May 1, 2012. 

The plaintiffs in this case were not among the ten SDOs named in the document. I 
packaged the 73 standards, the Notice of Incorporation, two posters, and other materials 
in 29-pound boxes and sent the boxes to the seven government officials and the ten 
SDOs. I sent the boxes by Federal Express on March 15, 2012. A copy of the Notice of 
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Incorporation memo may be found at https://law.resource.org/pub/us/cff/notice 
.sdo.20120315 to.pdf . 

22. After sending the standards, I received acknowledgements from several 
government addressees, including personal notes from the Chairman of the Federal Trade 
Commission, the Archivist of the United States, and the Chairman of the House 
Committee on Oversight and Government Reform. I did not receive any response from 
the SDOs. 

23. On May 1, 2012,1 posted the 73 documents on the Public Resource web 
site. I also began a process of examining the Code of Federal Regulations, the National 
Institute of Standards and Technology (NIST) database of Standards Incorporated by 
Reference (SIBR), and the Office of the Federal Register’s incorporation by reference 
listings to put together a list of documents that are incorporated into the CFR. I then 
began the process of trying to procure these documents, many of which are unavailable 
for purchase from the SDOs and which I had to obtain on the used book market. 

24. Every standard that I have posted on my website has been incorporated 
into law by a governmental authority. Public Resource does not impose any restrictions 
on the use of the standards. Public Resource has never charged for access to the standards 
or other legal materials, and has never asserted any intellectual property rights in them. 

We do not require people to log in or register before accessing content from Public 
Resource. 

25. Public Resource posted a PDF version of the 1999 Standards on its 
website. The PDF version accurately appeared as a scan of a physical version of the 
incorporated standard. Public Resource’s regular practice is to perform OCR on the 
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incorporated standards that it posts and to convert them further into standard Hypertext 
Markup Language (HTML) to make them still more accessible. I intended to do so for the 
1999 Standards, but I suspended further work on the 1999 Standards when this lawsuit 
was filed. In May 2014, Plaintiffs sued Public Resource for posting on its website and the 
Internet Archive website the 1999 Standards. Subsequently, so as to ensure that this 
lawsuit would be decided on a full record, in June 2014 Public Resource agreed to take 
down the versions of the 1999 Standards that it had posted on its website and on the 
Internet Archive website, pending the resolution of this case 

26. Public Resource has continued to develop techniques for making the 
documents that we post more usable, including double-keying and adding markup to 
HTML and SVG versions of the documents. Double-keying means having two separate 
typists copy the text of the incorporated standard; the results are then compared in order 
to eliminate any errors. We have also developed new markup techniques that increase the 
accessibility of the documents to people with visual impairments and print disabilities. 

We have also made significant advances in adding metadata to the documents, so each 
section, table, figure, and formula can be bookmarked and linked to, making internal 
navigation within the documents significantly friendlier for the user. 

27. We have applied these markup techniques to a number of standards 
incorporated by reference, though not to the 1999 Standards. Public Resource’s goal is to 
have the entire CFR, including all documents incorporated by reference, available in this 
new format so that users can seamlessly and transparently navigate the entire CFR. I 
believe this will be useful for employees of affected business enterprises, researchers and 
journalists covering public policy issues, government workers at the federal, state, and 
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local levels who must interact with the code as part of their daily activities, and for 
interested citizens. 

28. We have made several examples of our new approach available on the 
Internet and submitted them as examples of how the law can be made better in formal 
comments to Notices of Proposed Rulemaking that propose to incorporate standards by 
reference. 

29. Public Resource’s website is structured for navigation by search engines 
and for bulk access. Data are organized by country (e.g., /pub/us/) then by type of data, 
such as standards incorporated by reference (/pub/us/cfr/ibr/). 

30. Public Resource has one employee, myself, and three contractors who 
assist me in systems administration, conversion of graphics and formulas, and legal 
advice. Our core operating costs are under $500,000 per year, and we are funded entirely 
by donations, contributions and grants. Rather than adding staff, I have prioritized capital 
expenses, such as the purchase of the U.S. Court of Appeals backfile for $600,000 and 
the scanning of 3 million pages of Ninth Circuit briefs. Public Resource does not accept 
donations that are tied to the posting of specific standards or groups of standards. Public 
Resource’s operating income is not based on the amount of traffic its websites receive. 
Though we are a small organization, we observe all current best practices of corporate 
governance and transparency. I am proud that we have been awarded the GuideStar Gold 
Seal for nonprofit transparency. A full repository of our financials and other disclosures 
is maintained at https://public.resource.org/about . 

31. Public Resource has never sought benefit or compensation from its posting 
of the 1999 Standards. We have never used the 1999 Standards for marketing. 
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32. I pay a great deal of attention to quality control, including verifying the validity of 
the HTML, SVG, and MathML that I post. I respond immediately to any reports of errors 
from the public. 

33. To Public Resource’s knowledge, the 2014 edition of the Standards For 
Educational and Psychological Testing has not been incorporated by reference into law. 
Public Resource posts only those standards that have become law. Consistent with this 
policy, Public Resource has no plans to post the 2014 Standards on the Internet. 

34. My work at Public Resource, including the posting of standards incorporated by 
reference into federal and state law and my efforts to post briefs, opinions, regulations, 
statutes, and other materials that are edicts of government, are based on a long-held belief 
that the primary legal materials of our country must be available to all, especially those 
who lack the means to access the law in the status quo, because an informed citizenry is 
the key to the functioning of our democracy. 

I declare under penalty of perjury under the laws of the United States that the foregoing is 
true and correct. 

Executed this 21st day of January, 2016 at Sebastopol, California. 



Carl Malamud 


SF/5546571.2 
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Standards for Educational and Psychological Testing 
Sales Report, 1999 Edition 


Period 

Notes 

No. of 

Units 

FY 1999 

est. 

1,768 

FY 2000 

est. 

3,797 

FY 2001 

est. 

3,755 

FY 2002 

est. 

5,592 

FY 2003 

est. 

3,310 

FY 2004 

est. 

3,218 

FY 2005 

Actual 

3,803 

FY 2006 

Actual 

3,888 

7/1/06-12/31/06 

Actual 

2,144 

FY 2007 

Actual 

3,077 

FY 2008 

Actual 

3,358 

FY 2009 

Actual 

2,590 

FY 2010 

Actual 

3,043 

FY 2011 

Actual 

2,132 

FY 2012 

Actual 

1,649 

FY 2013 

Actual 

1,732 

FY 2014 

Actual 

855 

Total Units Sold 

49,710 
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Standards for Educational and Psychological Testing 
Sales Report 




FY 2000 
FY 2001 
FY 2002 
FY 2003 
FY 2004 
FY200S 
FY 2006 
7/1/06-12/31/06 
FY 2007 
FY 2008 
FY 2009 
FY 2010 
FY 2011 
FY 2012 
FY 2013 
Total Units Sold 


est. 3,797 

est. 3,755 

est. 5,592 

est. 3,310 

est. 3,218 

Actual 3,803 

Actual 3,888 

Actual 2,144 

Actual 3,077 

Actual 3,358 

Actual 2,590 

Actual 3,043 

Actual 2,132 

Actual 1.649 

Actual 1,732 

109,843 


est. 6,937 

est. 3,049 

est. 905 

est. 2,828 

est. 6,970 

est. 4,380 

est. 1,337 

est. 4,705 

est. 3,056 

Actual 1,324 


JA3084 


AE RA_AP A_N CME_0005137 









Case l:14-cv-00857-TSC Document 70-43 Filed 01/22/16 Page 1 of 2 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 63 of 441 


EXHIBIT 44 


JA3085 


Case l:14-cv-00857-TSC Document 70-43 Filed 01/; 
USCA Case #17-7035 Document #1715850 Filed: 0: 


Page 2 of 2 


Letters 

from page 4 

“Sex" refers to biology (.inti yes, as 
Anne Fausto-Stirling brilliantly 
recounts, the dividing people into two 
biological sexes rather than more, or a 
continuum, is hugely problematic), and 
“gender" is the whole set of attitudes, 
feelings, interests, clothing and behavior 
that has been arbitrarily divided into 
“masculine" and “feminine," causing 
profound damage when peoples atti¬ 
tudes, feelings and the rest are classified 
as either appropriate and normal or 
inappropriate and abnormal, depending 
on their biological sex. One way to pre¬ 
vent such damage is to make it crystal 
clear that ones sex should not be 
equared with and should not determine 
how one feels, thinks, dresses and acts. 
This cannor happen when the picture is 
blurred by die misuse of terms. 

For instance, when n journalist 
writes in “Both sexes seek attractiveness 
in one-night stand partners" (April, 
Monitor) that “both genders prioritise 


looks in their partners," the meaning 
is that people (of any sex) who arc 
feminine and those who are masculine 
prioritize looks in their parrners, not 
at all what was surely intended. This 
may be particularly confusing in light 
of research showing that both gay and 
straight men care more about parrners’ 
appearance than do women. 

PAOLA J. CAPIAN. PHD 
Cambridge, Mass. 

Evidence for the evolutionary? 

I APPRECIATE BEING PART OF A 
scientific, research-based profession. 
Therefore 1 am disrurbed when tread 
articles such as “Bonding over others’ 
business” by Zak Srambor in the April 
Monitor. At least 20 percent of the arti¬ 
cle was dedicated to speculating about 
the possible psychological behavior of 
our caveman ancestors. We have no 
data about this matter, and never can 
have it. The author and those he cites 
take our supposed reasons for gossip 
today, project them back on hypotheti¬ 
cal ancient conditions and say that is 



the reason for o _ 

we inventing a folklore of prehistory for 
ourselves? We would do better to limit 
ourselves to what is observable and 
tesrable. 

CAROLYN E. KERR, PHD 
Sevilla, Spain 


CORRECTIONS 

The roster of Centering on 
Mentoring cask force members in 
the May 2006 presidents column, 
listed Haydee M. Cuevas, PhD, as 
having obtained her degree in 1904. 
She received her degree in 2004. 

On page 35 of the June Monitor, 
Michael Schcier, PhD, is incorrectly 
listed as the speaker delivering the 
address, “Social and Psychological. 
Prcdicrors of Susceptibility to the 
Common Cold." Sheldon Cohen, 
PhD, is delivering the address. 
Scheier is chairing the session. 


Standards for Educational 
and Psychological Testing 

Developed jointly by the American Educational 
Research Association (AERA), the American 
Psychological Association (APA), and the National 
Council on Measurement in Education (NOME) 

The new Standards for Education and Psychological Testing 
has been revised significantly from the 1985 version with 
: more in-depth background material in each chapter, a 
greater number of standards, and a significantly expanded 
glossary and index. The new Standards reflects changes in 
federal law and measurement trends affecting validity: 

: testing individuals with disabilities or different linguistic 
backgrounds: and new types of tests as well as new uses of 
i. existing tests. This book is a vitally important reference for 
professional test developers, sponsors, publishers, users, 
policymakers, employers, and students in education and 
' psychology. 


_ Copies at $25.95 for members of: 

O AERA 
D APA. or 
□ NCME 

(please indicate your association! 

_ Copies at $31.95 List 

(for institutions as well as individuals 
who are not members of AERA, APA. or 
NCME) 

_ Shipping. US. Priority mail: 

$3 JO first copy: $1.50 e:ich add'l: 
noo-U.S. air mail: $7.50 first copy. S5.(X) 
each add’l 

Enclose check or money order made out 
to AERA and send order to: 

Test Standards, P.O. Box 465 
Hanover. PA 17331. 

To order by VISA or Mastercard, 
cull (800) 628-4094. 

Send fax orders and institutional purchase 
orders to (717) 633-8900. All orders must 
be prepaid. No returns. 


MONITOR ON PSYCHOLOGY 
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Standards for Educational 
and Psychological Testing 

l)tVEL0PLI> JOIXTIY BY AMERICAN EDUCATIONAL RESEARCH ASSOCIATION (ATRA). AMERICAN 
PSYCHOLOGICAL ASSOC’ATION (APA), .\AT10NAI. (JUJNC.I ON MTASUREWW IN EDUCA1 ION (NCMt) 


NEW! REVISED! EXPANDED! 

The hew Standards for Educational and Psychological Testing is 
now available! Revised significantly from the 1985 version, the 
1999 Standards has more in-depth background material In each 
chapter, a greater number of standards, and a significantly expand¬ 
ed glossary and index. The new Standards reflects changes-In fed¬ 
eral law and measurement trends affecting validity; testing Individu¬ 
als with disabilities or different linguistic backgrounds; and new 
types of tests as wed as new uses of existing tests. 

The Standards is written for the professional and for the educated 
layperson and addresses professional and'technical Issues of test 
development and use In education; psychology and employment. 

This book Is a vitally important reference for professional test devel¬ 
opers, sponsors, publishers, users, policymakers, employers, and 
students in education and psychology. The Standards has fifteen 
chapters organized Into three sections:- : - 


PARTI TESTCONSTRXTION, EVALUATION, 

AND DOCUMENTATION 

1. Validity 

2. Reliability arid Errors of Measurement 

3. Test Development and Revision 

4. Scales, Norms and Score Comparability 

5. Test Administration, Scoring, and Reporting '• .. 

6. Supporting Documentation for Tests 

PART II FAIRNESS IN TESTING 

7. Fairness In Testing and Test Use 

8. The Rights and Responsibilities of Test Takers 

9. Testing individuals of Diverse Linguistic Backgrounds. 

10. Testing Individuals with,Disabilities 

PART III TESTING .APPLICATIONS 

11 .The Responsibilities of Test Users 
•12. Psychological Testing arid Assessment 
.13. Educational Testing and Assessment 

14. Testing in Employment and Credehtlajiog 

15. Testing In Program Evaluation and Public Policy 


Order Form 

STANDARDS FOR EDUCATIONAL AND PSYCHOLOGICAL TESTING 

_Copies at $25.95 for members of 

AERA, APA, or NCME only. 
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$ 

n Check or money order enclosed, made out to AERA. 

□ Charge my □ VISA or □ Mastercard 
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Siooature 


Name 

_Copies at $31.95 List (for 

$ 

Address 

institutions as well as individuals 



who are not members of AERA, 
APA or NCME) 


Cto State- Zip. 


Phone 

(U S. Priority mail: $3.50 (list copy, $1.50 
each additional copy ip to 9 copies. 10 or 
more copies ship by UPS; call for charges) 

SUBTOTAL 

$ 

Institutional purchase orders must be sent to AERA Publications Sales,' 

1230 17th St. NW. Washington. DC 20036. 

Institutional Purchase Order No. 

(Nor-U.S. air malt $.7.50 first copy, $5.00 
each adrStkxial copy.) 

SHIPPINGS. HANDLING 

$ 

TOTAL AMOUNT ENCLOSED 

s 

All orders must be prepaid, lb order by VISA or Mastercard, call 
(800) 628-4094. No returns. (Prices subject to change without notice.) 
Send orter to: Test Standards, P.O.Box 465, Hanover, PA 17331. 
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STANDARDS 



for educational and psychological testing 


Developed jointly by the American Educational Research Association, the American 
Psychological Association, and the National Council on Measurement in Education, 
Standards for Educational and Psychological Testing addresses professional and tech¬ 
nical issues of test development and use in education, psychology, and employment. 
The current edition includes changes in federal law and measurement trends affecting 
validity, testing individuals with disabilities or different linguistic backgrounds, and 
new types of tests, as well as new uses of existing tests. Now in its fifth printing since 
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Developed jointly by 
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National Council on Measurement in Education (NCME) 

The new Standards for Educational and Psychological Testing is now available! Revised 
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glossary and index. The new Standards reflects changes in federal law and measurement 
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The 2014 edition of the Standards for Educational and Psychological Testing is now on sale. The 
Testing Standards are a product of the American Educational Research Association, the American 
Psychological Association (APA), and the National Council on Measurement in Education (NCME). 
Published collaboratively by the three organizations since 1966, it represents the gold standard in 
guidance on testing in the United States and in many other countries. Rend more 
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Plaintiffs/Counter-defendants, 
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Defendant/Counterclaimant. 


EXPERT REPORT OF JAMES R. FRUCHTERMAN 


JA3115 





Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 3 of 127 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 94 of 441 

Table of Contents 

Introduction......................1 

Background and Qualifications.2 

What Does Accessibility Mean for a Person Who is Blind?.3 

Locating an Accessible Version of the 1999 Standards.5 

Testing the Public.Resource.Org Website's Accessibility.7 

Making the 1999 Standards Accessible Today.8 

The Public.Resource.Org Version of the 1999 Standards.9 

Confirming Accessibility for People Who Are Blind.9 

The Archive.org Version of the 1999 Standards.11 

Conclusion................12 


JA3116 












Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 4 of 127 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 95 of 441 

Introduction 

As an expert in accessibility of written materials for people who have disabilities that 
affect using standard print (people who are print disabled), I have been retained by 
Public.Resource.Org to evaluate the accessibility of certain content that had been available on 
the website of the defendant in this case. As someone dedicated to improving accessibility for 
the benefit of people with disabilities and in the public interest, I agreed to evaluate the 
accessibility to people who are blind of this specific commonly used standard document. 

This expert report is a summary of certain opinions that I intend to give, if asked, at trial 
regarding the accessibility of specific documents to people who are blind or print disabled. This 
report also states the bases for my opinions, and it discloses the data or other information 
considered in forming those opinions. I reserve the right to change or supplement this report if 
additional evidence comes to my attention, and to prepare demonstratives and/or exhibits to 
illustrate or explain my opinions, as appropriate. 

A copy of my curriculum vitae, including a list of my publications and presentations, is 
Exhibit A to this report. I provide my expertise in this case pro bono, and I am not receiving 
compensation for my time researching, writing this report, or testifying. I previously served as 
an expert in The Authors Guild, Inc. et al. v. HathiTrust, et al.. Case No. 1:1 l-cv-06351-HB 
(S.D.N.Y.) (case filed September 12, 2011) and I am serving as an expert in American Society of 
Testing and Materials, et al. v. Public.Resource.Org, Case No. l:13-cv-01215-TSC-DAR, 
although I have not testified in either case. I have not given deposition or trial testimony in the 
past four years. 
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Background and Qualifications 

I serve as Founder and Chief Executive Officer of Benetech, a nonprofit dedicated to 
creating new technology solutions that serve humanity and empower people to improve their 
lives. In 1980 I earned a B.S. in Engineering and an M.S. in Applied Physics from California 
Institute of Technology. I co-founded Calera Recognition Systems in 1982. Calera developed 
optical character recognition (OCR) technology that allowed computers to read virtually all 
printed text. 

In 1989,1 founded Arkenstone, a nonprofit social enterprise, which produced reading 
machines for the print disabled community based on the Calera technology, and was at one time 
the largest maker of affordable reading systems for the blind. The Arkenstone product line was 
sold in 2000 and the resulting capital funded the next phase of Arkenstone under its new name, 
Benetech. I have been the CEO of Benetech/Arkenstone since 1989. 

I have served on three U.S. federal government advisory committees for disability issues: 
the Section 255 Telecommunications Access Advisory Committee, the Section 508 Electronic 
Information and Technology Access Advisory Committee, and the Advisory Commission on 
Accessible Instructional Materials in Postsecondary Education for Students with Disabilities. 

I have received numerous other awards and recognition for my work making print materials 
accessible to people who are blind or otherwise print disabled. In 2006 I received a MacArthur 
Fellowship. I was named an Outstanding Social Entrepreneur in 2003 by the Schwab 
Foundation and have frequently participated in the World Economic Forum Annual Meetings in 
Davos, Switzerland. Benetech received the Skoll Award for Social Entrepreneurship under my 
leadership. I also received the Migel Medal from the American Foundation for the Blind, the 
Robert F. Bray Award from the American Council of the Blind, and the American Library 

2 


JA3118 


Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 6 of 127 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 97 of 441 

Association’s Francis Joseph Campbell Award in recognition of my successful efforts to make 
literary works more accessible to people who are blind or visually impaired. 

What Does Accessibility Mean for a Person Who is Blind? 

Accessibility is usually defined in a functional way: can a person with a disability 
independently access the same information and perform the same tasks as a person without a 
disability? When it comes to accessing materials traditionally available as print, such as 
standards, there are many groups of print disabilities. The most severe is blindness, where a 
person cannot perceive the printed text at all. The next is vision impairment, where a person 
generally cannot perceive the text directly or with corrective lens, but may be able to use 
magnifiers of different types to read the text. Another group is learning disabilities that interfere 
with reading, such as dyslexia. A closely related group of disabilities involve brain injuries that 
affect reading or the retention of material read. Another group is physical disabilities that 
interfere with the holding or seeing of books or the turning of pages. 

In this report, I focused on the accessibility challenges that would be experienced by 
blind people, because they are generally the most severe print disabilities. The other groups of 
people with print disabilities use similar technologies to access print (such as having it read 
aloud), and experience similar challenges as blind people. In the accessibility field, it is 
generally understood that if you make information accessible to a blind person, it will probably 
also meet the accessibility needs of the great majority of people with other print disabilities. 

The most common technology used by a blind person for accessibility is called a screen 
reader. As the name suggests, a screen reader is a program that runs on a personal computer or a 
smartphone that reads the information on the screen aloud (using a computer-synthesized voice) 
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to a blind person. The screen reader runs “on top of’ other programs, figuring out not only what 
text is on the screen, but also the controls that are displayed: items such as buttons, menus, text- 
entry boxes and the like. Because of the amount of information on a complete screen, and its 
complexity, blind people need to be able to focus on the most important information so that they 
do not waste time listening to everything on the screen. 

For the purpose of this report, measuring the accessibility of standards, I am assuming 
that the blind user is using a screen reader on top of a web browser or word processor program 
on a personal computer. Based on the information the screen reader can glean from the pages 
displayed on the screen, can a blind person locate the standard and read it? 

The accessibility tasks I tested were designed to assess whether a blind user with basic 
assistive technology skills could perform the same kind of tasks one might expect a user without 
a disability to perform in accessing a given standard, without requiring the intervention of a third 
party. This functional approach is the most common method of assessing accessibility. 

The specific tasks I investigated were: 

• Could a blind user with basic assistive technology skills independently access a 
specific standard of interest? 

• Could a blind user independently read the entire standard using assistive technology? 

• Could a blind user independently navigate to a specific place in the standard and read 
the content in that place? 

• Could a blind user independently do a full text search and find specific mentions of 
terms of interest? 
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I conducted these tests on a standards document that was represented to me as having been 
available on the Public.Resource.Org website. I primarily used the Window-Eyes screen reading 
software and the ABBYY FineReader optical character recognition software to perform my tests. 

Locating an Accessible Version of the 1999 Standards 

I was asked to review the accessibility of the 1999 edition of The Standards for 
Educational and Psychological Testing (hereafter, the “1999 Standards”) for people who are 
blind or otherwise print disabled. The first step in determining the accessibility of a document is 
to try to locate a version of the 1999 Standards that would be accessible to people who are blind 
or have print disabilities. I attempted to locate an accessible version of the 1999 Standards 
through two separate avenues: by searching the catalogs of the main libraries that serve people 
with print disabilities, and also by doing a standard Google search to try to locate an electronic 
version of the 1999 Standards. From my work with people who are blind or print disabled, I 
know that this would be the typical procedure that people who are blind or print disabled would 
perform when looking for an accessible version of a document. 

The four main libraries that serve people with print disabilities are the American Printing 
House for the Blind, Bookshare (which I founded), Learning Ally, and the National Library 
Service for the Blind and Physically Handicapped, Library of Congress. I performed a thorough 
search of all four of these catalogs and found that the 1999 Standards were not available through 
any of these resources, either in an electronic form, or in mail-delivery braille or audio recording. 

I then performed a Google search to attempt to locate an electronic version of the 1999 
Standards online. I was unable to find an electronic version of the 1999 Standards online, but I 
did locate a used print version for sale on Amazon.com. I have been informed by counsel for 
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Public.Resource.Org that although Pub lie.Resource. Org previously hosted an electronic version 
of the 1999 Standards on its website, it had been taken down during the course of this litigation. 
From my research I believe that a version of the 1999 Standards that is accessible to people who 
are blind or print disabled is currently unavailable to the public. 

The unavailability of a version of the 1999 Standards that is accessible to people who are 
blind or print disabled is problematic because the 1999 Standards are important references for 
those making tests that are accessible to students who are print disabled, as well as those 
impacted by these tests. For instance, the 1999 Standards were referred to in several works 
concerning test accessibility for blind students, specifically: Test Access: Making Tests 
Accessible for Students with Visual Impairments: A Guide for Test Publishers, Test Developers, 
and State Assessment Personnel, Second Edition, by Carol B. Allman, Ph.D., published by the 
American Printing House for the Blind (Exhibit C) , and an online resource published by the 
American Foundation of the Blind, Building Assessment Initiatives for Schools: Guidelines to 
Support the Contract Development Process Between Test Publishers and States (Exhibit D) . As 
an expert in the field, this means that the 1999 Standards are important references today for those 
making tests accessible to students with disabilities such as blindness. This also means that it is 
an important resource to any students or other individuals with print disabilities that want to 
assess compliance with the 1999 Standards. The unavailability of the 1999 Standards means that 
some of those who are most impacted, people who are blind or print disabled, are unable to 
independently access the 1999 Standards. 
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Testing the Public.Resource.Org Website's Accessibility 

Because the 1999 Standards are no longer hosted on the Public.Resource.Org website 
during the course of this litigation, I was not able to locate the full text of the 1999 Standards on 
the Public.Resource.Org website while performing my Google search referenced above. 

However, searching the terms “1999 Standards for Educational and Psychological Testing” on 
Google (for me) shows the page where the 1999 Standards had been located on the 
Public.Resource.Org website, located in the first page of links in the search results. However, 
the file I found there was a placeholder noting the voluntary takedown of the file. I have also 
searched for other standards that Public.Resource.Org has posted on its website, such as NFPA 
101-2000, and I have found that it would be relatively easy for a person who is blind or print 
disabled to use screen reader software and perform a Google search to locate a standard if it was 
available on Public.Resource.Org’s website. Therefore, when the 1999 Standards had been 
hosted on the Public.Resource.Org website, a person who is blind would have been able to locate 
the 1999 Standards through a simple Google search, with the assistance of screen reader 
software. 

The Public.Resource.Org website has no required sign-up procedure. It is possible to go 
directly to a specific standard either by using a direct weblink or by navigating the text-oriented 
website. This is important because sign-up procedures can often have the effect of preventing 
people who are blind or print disabled from accessing certain parts of websites due to the fact 
that many sign-up procedures use unlabeled buttons or other elements that screen reader software 
ca nn ot read. Therefore, a person who is blind or print disabled would have been able to locate a 
version of the 1999 Standards on the Public.Resource.Org website when it was still hosted there, 
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and that person would then have been able to gain access to that electronic version of the 1999 
Standards. 

Making the 1999 Standards Accessible Today 

Because an accessible version of the 1999 Standards is not currently available, if a blind 
person needed to have an accessible version of the 1999 Standards, they would need to create it 
themselves or request that their employer, educational institution, or a specialized library for the 
blind create it. Generally, most blind people themselves do not have the ability to convert books. 
Some blind people have their own home scanners, and if they purchased a used copy online, 
would be able to scan the 1999 Standards page by page on a home scanner, which would take at 
least two hours of labor, and then perform optical character recognition on the title. Optical 
character recognition is the process by which a computer converts images of printed text into 
machine-encoded text that can be read aloud by a screen reader. If the scanning quality wasn’t 
very good, significant numbers of errors would be introduced through the optical character 
recognition process. The resulting word processor file of recognized text could then be read 
using a screen reader. 

If Bookshare were to make the 1999 Standards accessible to a blind person, we would 
purchase a used copy of the printed version, chop off the bindings and then process it through a 
high speed scanner to obtain a high quality scan of the book in less than fifteen minutes. We 
would then perform optical character recognition on the image scans of all of the pages of the 
book, which typically creates a Microsoft Word file version of the text, and then send it to an 
outside service (or a volunteer) to have it proofread, correcting errors introduced by the 
limitations of optical character recognition. Public.Resource.Org has already performed the 
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great majority of the most expensive and time consuming steps needed to create an accessible 
version of this document, specifically purchasing a print version of the title, waiting a few days 
to receive it, chopping off the binding and scanning it with a high speed production scanner, or 
utilizing a library-grade nondestructive book scanner. This is a valuable contribution to anyone, 
individual or organization, that wanted to ensure that the 1999 Standards are accessible to people 
who are blind or print disabled, if that file were still available. 

The Public.Resource.Org Version of the 1999 Standards 

I was supplied with a version of the 1999 Standard in PDF format. It was represented to 
me that this file had been available online at the Public.Resource.Org website. I examined the 
file, and found it to be a high quality image scan of the 1999 Standards. If the file was still 
online, this would have meant that a blind person wanting to have an accessible version of the 
1999 Standards would be able to do so by performing optical character recognition on the 
Public.Resource.Org image file, creating an accessible text version of the 1999 Standards in 
minutes. 

Confirming Accessibility for People Who Are Blind 

I then performed the steps of taking the Public.Resource.Org version of the 1999 
Standards and making it accessible, while using Window-Eyes screen reading software to read 
the words on the computer screen aloud. For the version of the 1999 Standards that was on the 
Public.Resource.Org website, I used ABBYY FineReader optical character recognition software 
to recognize page images, and it converted those pages into a Microsoft Word document. In 
addition, the process of using the ABBYY software and reading the document was something a 
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blind person could do independently using Window-Eyes software to perform the tasks in an 
accessible way, because the program speaks the menus and converted text aloud. Because the 
image scan by Public.Resource.Org was high quality, there were few optical character 
recognition errors. In addition, I also tested a typical page image from the Public.Resource.Org 
version using the website Free Online OCR (http://www.onlineocr.net/) , and confirmed that it 
also recognized the text well. In my opinion, most of the OCR solutions that would be available 
to people who are blind should be able to convert this image PDF document into accessible text. 

a.H O i- l ■ I .j'c j:e_'iuchteiman report 201S-06-XK doc, ■ Microsoft Word - 0 SS 

JS % fr m U ;u u & J> 



I then examined in Microsoft Word several pages of the standard as processed by 
ABBYY FineReader, and confirmed that Window-Eyes could read the text aloud in logical 
reading order. I also successfully performed full text searches on a key word, a standards 
number, and a page number, using Window-Eyes. 
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My tests therefore indicated that a blind person using a screen reader would be able to 
perform all of the functional tasks: reading the entire standard, navigating to a specific place in 
the standard, or searching on key terms. Because the text is provided in a standard format, such 
as Microsoft Word, a blind person is able to listen to the text, or access it using a digital braille 
device. This kind of text content is also highly accessible to people with other print disabilities 
and the assistive technology they use to access print. For example, people with low vision or 
with dyslexia often use a screen reader to read text aloud. 


The Archive.org Version of the 1999 Standards 

I was also supplied with a version of the 1999 Standard in TXT (text) format, by a staff 
person at the Internet Archive, operator of the Archive.org website. It was represented to me by 
this person that this file, aera.standards.l999_djvu.txt, had been available online at the 
Archive.org website. According to the Internet Archive’s “Derivatives” page located at 
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https://archive. org/help/derivatives .php , when a PDF file is uploaded to the Internet Archive 
website, the website automatically creates derivative file types that are also accessible on that 
website, including TXT format. The deposition testimony of Christopher Butler from the 
Internet Archive, as well as the deposition testimony of Carl Malamud from Public.Resource.Org 
indicate that when Public Resource uploaded the PDF file of the 1999 Standards to the Internet 
Archive website, the Internet Archive automatically created this text file of the 1999 Standards, 
which was publicly accessible on the Internet Archive website. 1 I examined the file, and found it 
to be a text version of the 1999 Standards, preceded by informational material about the Internet 
Archive in HTML format. It appeared to me that the text version had been created by optical 
character recognition, because there were a few uncorrected errors typical of that process. 

As established in my discussion of the 1999 Standards on the Public.Resource.Org 
website above, once the 1999 Standards are available in an electronic text format, a blind person 
using a screen reader would be able to perform all of the functional tasks: reading the entire 
standard, navigating to a specific place in the standard, or searching on key terms. I confirmed 
that this was the case with the aera.standards.l999_djvu.txt file. Because the text is provided in 
a standard and compatible format, a blind person is able to listen to the text, or access it using a 
digital braille device. This kind of text content is also highly accessible to people with other 
print disabilities and the assistive technology they use to access print. 

Conclusion 

I was asked to review the accessibility of the 1999 edition of The Standards for 
Educational and Psychological Testing. I was unable to find an accessible version of the 

1 Deposition of Christopher Butler of the Internet Archive, December 2, 2014, at pp. 48-49, 87, 102-105; 
deposition of Carl Malamud of Public.Resource.Org, May 12, 2015, at pp. 281-284. 
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document online. If the document provided to me by Public.Resource.Org had been online on 
their website, I believe a blind person of ordinary technical skill would have been able to 
independently use that document and commonly available optical character recognition 
technology to create an accessible version of the 1999 Standards, and carry out reading and 
reference tasks similar to those a person without a disability would be able to do with a print 
version of the standard. If the document provided to me by Archive.org had been online on their 
website. I believe a blind person of ordinary technical skill would have been able to 
independently use that document directly to carry out reading and reference tasks similar to those 
a person without a disability would be able to do with a print version of the standard. 


Dated: June 13. 2015 
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James R. Fruchterman 

Founder and CEO 
Benetech 


Education 

■ California Institute of Technology 
B.S. Engineering, 1976-80 

M.S. Applied Physics, 1978-80 

■ Stanford University, 1980-81 

Ph.D. Studies in Electrical Engineering 

Professional Experience 

■ CEO and Founder, 2015-present 

President, CEO, Chairman, Founder, 2000-2014 
Benetech (name changed from Arkenstone in 2000) 
Palo Alto, California 

■ President, CEO, Chairman, Founder, 1989-2000 
Arkenstone, Inc. 

Moffett Field, California 

■ Director, 1989-present 

Vice President Finance, CFO, 1989-2004 
President & CEO, Founder, 1989-95 
RAF Technology, Inc. 

Palo Alto, California and Redmond, Washington 

■ Vice President, Marketing, 1987-89 
Founder, Vice President, Finance, 1982-88 
Calera Recognition Systems, Inc. 

Santa Clara, California 

■ Prior engineering positions with: 

■ Phoenix Engineering, Inc. 

■ G.C.H., Inc. 

■ IBM T.J. Watson Research Center 

■ General Motors Company 

■ NASA — Jet Propulsion Laboratory 

■ Fermi National Accelerator Laboratory 
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Publications 

■ Technology Serving Humanity (chapter). In Schultz, R. (editor) Creating Good Work. Palgrave 
Macmillan, February 2013 

■ Guest Editor’s Page. AFB Journal of Visual Impairment & Blindness, October-November 

2012 

■ An Interview With Technology Gum George Kerscher, AFB Journal of Visual Impairment 
& Blindness, October-November 2012 

■ For Love or Lucre. Stanford Social Innovation Review, Spring 2011 

■ Developing Information Technology to Meet Social Needs . In Innovations , MIT Press, 2008 

■ Accessing Books and Documents, a chapter in the book, Assistive Technology for Vision- 
Impaired and Blind People . Springer Verlag 2008 

■ Everyone Deserves Access to Technology. OpEd in The Sacramento Bee by Jim Fruchterman 
and Gregg Vanderheiden, June 17, 2007 

■ Document Recognition Serving People With Disabilities, Proc. SPIE 6500, International 
Society for Optics and Photonics, 2007 

■ Pattern Recognition Technology Helps Disabled People Access Books. SPIE Newsroom, 
International Society for Optics and Photonics, May 14, 2007 

■ Nothing Ventured Nothing Gained. Addressing the Critical Gaps in Risk-Taking Capital for 

Social Enterprise, by Jed Emerson, Tim Freundlich and Jim Fruchterman, published by 
Oxford Said Business School, 2006 

■ Build Great Companies. Then Help Build a Great World. OpEd in The San Jose Mercury News, 
November 13, 2006 

■ Comments on Accessibility of Google Print and Google’s Library Project, white paper, 
February 2005 

■ Technology Benefiting Humanity, published in the Association for Computing Machines 
Ubiquity magazine, March 2004 

■ The Power of Technology Social Enterprises, published in the N-TEN forecast series, 
February 2004 

■ In the Palm of Your Hand: A Vision of the Future of Technology for People with Visual 

Impairments, published in the American Foundation for the Blind’s Journal of Vision 
Impairment and Blindness . October 2003 

■ The Chafee Amendment: Improving Access to Information, published in Information 
Technology and Disabilities . a journal published by Equal Access to Software and Information 
(EAST, co-authored with Bookshare Senior Product Manager Alison Lingane, October 2003 

■ The Soundproof Book: Exploration of Rights Conflict and Access to Commercial EBooks 

for People with Disabilities, published in First Monday, co-authored with George Kerscher, 
the International Project Manager of the DAISY Consortium. May 2002 

■ Bookshare. Books without Barriers , at the Closing the Gap conference, Minneapolis, MN, 
October 2001 
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■ Two presentations given at the IT Accessibility 2001 Conference, May 2001 at the National 
Institute of Standards and Technology 

■ I Dream of Software 

■ The Business Case for Adaptive Technology 

■ Humanizing the Voice of the Machine, with Prof. Mari Ostendorf (University of 
Washington), Annual Meeting of the American Association for the Advancement of the 
Machine, Boston, MA, February 2000 

■ The Many Facets of Open Book: Ruby Edition . California State University, Northridge 
(CSUN), 15th Technology and Persons with Disabilities Conference, March 2000 

■ Corporate Responsibility for Adaptive Technology. California State University, Northridge 
(CSUN), 14th Technology and Persons with Disabilities Conference, March 1999 

■ Developing Partnerships for Assistive and Universally Designed Technology for Persons 

with Disabilities. Testimony before United States House of Representatives, Committee on 
Science, Subcommittee on Technology, August 4, 1998 

■ Access to Maps and Location Information through Virtual Reality Techniques and GPS 
Satellite Receivers, 3rd International Technical Aids Seminar, Tokyo, Japan, July 1994 

Invited Talks 

■ “ Innovation in America: The Role of Technology August 1, 2013, Testimony before U.S. 
House of Representatives, Judiciary Committee’s Subcommittee on Courts, Intellectual 
Property, and the Internet. 

■ “Social Change at Scale - That’s Innovation!” May 2012, TEDxSanJoseCA 2012, San Jose, 
CA. 

■ ‘The Power of Failure, People and Karma Ranking, ’’ May 20, 2012, Commencement speech, St. 
Mary's College, Moraga, CA. 

■ “Raising the Floor, ” October, 2011, Keynote Speech, Association for Education and Rehabilitation 
of the Blind and Visually Impaired Conference, Cleveland OH. 

■ Keynote speech, IEEE Sections Congress, August 2011, San Francisco, CA. 

■ “ Making the Book Truly Accessible. ” Tools of Change Conference, New York, NY, 2011 
Keynote Speech 

■ UBS-Ashoka Visionaris Award, Keynote Speech, Social Entrepreneur of the Year Award, 
Mexico City, Mexico, September, 2010 

■ A series of three invited speeches on Bookshare and accessible books, in Tokyo, Shizuoka 
and Osaka, Japan, February, 2009 

■ Keynote Speech, Social Enterprise World Forum, Edinburgh, Scotland, September, 2008 

■ “Raising the Floor: Providing Accessible Technology and Content to Every Person with a 
Disability on the Planet,” International Conference on Computers Helping People with 
Special Needs, Linz, Austria, July, 2008 Keynote Speech 

■ “Raising the Floor,” CSUN Conference on Technology and Persons with Disabilities, 
March, 2008 Keynote Speech 
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■ Extensive speaking engagements to students about technology serving people with 
disabilities. Have done invited talks at: 

■ Stanford University 

■ University of California at Berkeley 

■ Brigham Young University 

■ University of the Pacific 

■ Santa Clara University 

■ California Institute of Technology 

■ San Jose State 

■ University of California at Santa Cruz 

■ University of California at Davis 

■ Loyola Marymount University 

■ Pepperdine University 

■ University of Washington 

■ Columbia University 

■ Harvard University 

■ University of Geneva 

■ Oxford University 

■ Inflection Point Opportunities in Social Investment, Closing Keynote for the UBS 
Philanthropy Forum, Lisbon, Portugal, July 2007 

■ It’s Not Rocket Science: Building Social Enterprises, Keynote for the 7th Gathering of the 
Social Enterprise Alliance, Adanta, Georgia, March 2006 

■ Opening Keynote for the Global Social Venture Competition, New York, April, 2006 

■ Keynote for the 7th IAPR Workshop on Document Analysis Systems, Nelson, New 
Zealand, February 2006 

■ Building a Global Library for People with Print Disabilities, a speech for the World Summit 
on the Information Society, Tunis, Tunisia, November 2005 

■ Innovating Information Technologies to Protect Human Rights, a speech for the World 
Affairs Council of Northern California, February 2004 

■ Setting the 2004 Agenda: Technology, speaker at the World Economic Forum, Davos, 
Switzerland, January 2004 

■ Seizing Market Failure as an Investment Opportunity, Keynote for the Business for Social 
Responsibility Annual Conference, Los Angeles, November 2003. 

■ In the Palm of Your Hand, Keynote for the World Blind Union Asia Pacific conference, 
Singapore, November, 2003 

■ Technology and Human Rights, University of Peradeniya, Sri Lanka, November, 2003 

■ When Markets Fail, Who Responds? Discussion Leader at the World Economic Forum, 
Davos, Switzerland, January 2003 

■ Technology for Nonprofits, with Michael Gilbert, National Gathering for Social 
Entrepreneurs, Minneapolis, MN, December, 2002 

■ Bookshare: Large Scale, Web-Based Accessible Books, TechShare conference organized by 
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the Royal National Institute of the Blind, Birmingham, UK, November 2002 

■ Putting Technology to Work for Development, speech at the United Nations to the joint 
meeting of the World Technology Network and UNOPS, July 2002 

■ Bookshare: The Project for Creating Accessible Books through Computers, at the General 
Session of the National Federation of the Blind 2002 Annual Convention, July 2002 

■ Stanford Social Entrepreneurship Conference, January 2002 

■ The Once and Future Web: Tenth Anniversary of the First U.S. Web Page at the Stanford 
Linear Accelerator Laboratory, December 2001 

■ Netlmpact Annual Conference at Kenan-Flagler Business School, November 2001 

■ American Council of the Blind Annual Convention, July 2001 

■ Bringing Socially Beneficial Technology into the Service of Humanity . EE380 at Stanford 
University, April 2001 

■ Information Technology in the Service of Human Rights at the Computers, Freedom and 
Privacy Conference, March 2001 

■ Rank Prize Fund Symposium, Grasmere, England 

■ Guest Lecturer for CSUN program in disability leadership 
Professional Associations 

■ Association for Computing Machinery 

■ Institute of Electrical and Electronics Engineers 

■ American Association for the Advancement of Science 

■ Social Enterprise Alliance 
Awards and Public Service 

■ Head of Benetech Delegation, Diplomatic Conference to Conclude a Treaty to Facilitate 
Access to Published Works by Visually Impaired Persons and Persons with Print Disabilities, 
World Intellectual Property Organization, Marrakesh, Morocco (2013) 

■ Member, Global Agenda Council on Measuring Sustainability, World Economic Forum 
(2012-2014) 

■ Member of the Board of Directors, ZeroDivide, foundation investing in community 
enterprises that leverage technology to benefit people in low-income and other underserved 
communities (2007-2013) 

■ Commissioner, Federal Advisory Commission on Accessible Instructional Materials in 
Postsecondary Education for Students with Disabilities, 2010-2011 

■ Duke University, CASE Award for Enterprising Social Innovation, 2011 

■ Brigham Young University, Center for Economic Self-Reliance Social Innovator of the Year, 
2009 

■ AT&T Technology Innovation Award from the Alliance for Technology Access, March 
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2008 

■ Strache Leadership Award from the California State University, Northridge, 2007 

■ John D. and Catherine T. MacArthur Foundation Fellowship, 2006 

■ Technical Advisory Committee Member, National Instructional Materials Accessibility 
Standard, U.S. Department of Education (2005-2008) 

■ Advisory Committee Member, National Instructional Materials Accessibility Center, U.S. 
Department of Education (2006-present) 

■ Skoll Award for Social Entrepreneurship. 2004 and 2006 

■ Fast Company Social Capitalist Award: Top 20 Groups Changing the World, 2004 

■ Laureate, The 2003 and 2001 Tech Museum Awards 

■ American Library Association Francis Joseph Campbell Award. 2003 

■ Schwab Foundation Outstanding Social Entrepreneur of 2003 Award 

■ Member, the Community Partnership Committee, which oversees a diversity and disability 
agreement with SBC, Inc. 

■ Runner-up, Yale-Goldman Sachs National Nonprofit Business Plan Competition, 2003 

■ American Foundation for the Blind Access Award, 2003 

■ Robert S. Bray Award. The American Council of the Blind 

■ Winner, Education Category, 2002 Stockholm Challenge 

■ Fast 50 Champion of Innovation 2002 

■ Judge, 2002 National Social Venture Competition 

■ Member, Board of Directors of the Social Enterprise Alliance (2000-2010, chair 2008-2010) 

■ Member of the Advisory Board, Telecommunications Access Rehabilitation Engineering 
Research Center, a joint effort of the Trace R&D Center of the University of Wisconsin- 
Madison and the Technology Access Program of Gallaudet University, 2001 

■ Panelist, National Science Foundation Small Business Innovation Research Program, 1998, 
2000, 2003 

■ Participant, 1998 NSF Workshop for Discussing Research Priorities and Evaluation 
Strategies in Speech Synthesis, August, 1998 

■ Member, Electronic Information and Technology Access Advisory Committee, a federal 
advisory committee responsible for drafting federal acquisition standards for accessibility 
under Section 508,1998-1999 

■ Member, Telecommunications Access Advisory Committee, a federal advisory committee 
responsible for making recommendations to the U.S. Access Board and Federal 
Communications Commission on implementing portions of the 1996 Telecommunications 
Act, 1996-1997 

■ U.S. Patent Number 5,470,223: System and Method for Tracking a Pedestrian 
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■ Finalist, 1996 Discover Magazine Awards for Technological Innovation 

■ 1996 Access Award, American Foundation for the Blind 

Major Works and Areas of Expertise 

■ Founder and CEO of Benetech. a highly innovative nonprofit company focused on using 
the power of technology to address social needs in areas such as disability, literacy, human 
rights and the environment. 

■ Founder of Arkenstone, Inc., a leading nonprofit organization providing adaptive 
technology for education and employment for people with disabilities and the largest maker 
of reading systems for people with blindness, vision impairment and learning disabilities. 
Developer of the Arkenstone Reader, the first affordable reading system for the blind. 
Designer of Open Book, the first talking Windows program for the blind. Co-inventor of 
Adas Speaks, the first accessible map software for the blind, and of Strider, a talking GPS 
locator for the blind. 

■ Cofounder of RAF Technology. Inc., the nation's leading company in optical character 
recognition technology for processing forms in postal and medical applications. RAF's 
software is used to route the United States mail. 

■ Cofounder of Calera Recognition Systems, Inc., the first company to develop omnifont 
optical character recognition that works without user training. 
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Exhibit B 


Documents, Facts, or Data Considered in Forming My Opinions: 

• The Public.Resource.Org website, at www.public.resource.org 

• The American Printing House for the Blind website, at www.aph.org/ 

• The Bookshare website, at www.bookshare.org/cms 

• The Learning Ally website, at www.leamingally.org 

• The National Library Service for the Blind and Physically Handicapped, Library of 
Congress website, at http://www.loc.gov/nls/catalog/?loclr=blognls 

• Test Access: Making Tests Accessible for Students with Visual Impairments: A Guide for 
Test Publishers, Test Developers, and State Assessment Personnel, Second Edition, by 
Carol B. Allman, Ph.D., published by the American Printing House for the Blind, and an 
online resource published by the American Foundation of the Blind, available at 
www.aph.org/tests/access2 

• Building Assessment Initiatives for Schools: Guidelines to Support the Contract 
Development Process Between Test Publishers and States, available at 
www.afb.org/info/afb-national-education-program/iltli-2005-education- 

summarv/checklist-for-rfp-building/235 

• Window-Eyes screen reader software 

• ABBYY FineReader optical character recognition software 

• The 1999 Standards in image only PDF format, as provided to me by 
Public.Resource.Org 

• The 1999 Standards in TXT format, as provided to me by the Internet Archive 

• The deposition of Christopher Butler of the Internet Archive, December 2, 2014 

• The deposition of Carl Malamud of Public.Resource.Org, May 12, 2015 
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Test Access 

Making Tests Accessible for 
Students with 
Visual Impairments: 

A Guide for Test Publishers, 

Test Developers, and 
State Assessment Personnel* 


A 

Large Print 



>»i) 

Audio 
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TEST ACCESS: 

Making Tests Accessible 
for Students With 
Visual Impairments 

A Guide for Test Publishers, 
Test Developers, and State 
Assessment Personnel* 


Fourth Edition 


Carol B. Allman, Ph.D. 


Published by 

American Printing House for the Blind 
Louisville, Kentucky 
July 2009 

AMERICAN PRINTING HOUSE 
FOR THE BLIND, INC. 
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*Book Number two in the TEST ACCESS Series, promoting 
accessibility of testing materials for persons who are blind or 
visually impaired 


©2004, 2007, 2008, 2009 American Printing House for the Blind, 
Inc. With the exception of ETS Guidelines for a Test Reader, which is 
material presented in Appendix G, this document may be copied in 
whole or in part and distributed free of charge for educational and 
nonprofit use as long as appropriate credit is given to the author and 
publisher, and the "Work in Progress" notice is included on each copy. 
No other use of this material is allowed without written permission. 

Work in Progress: This document represents a set of guidelines 
for making tests accessible to students with visual impairments. 
These guidelines are a "work in progress" and will be routinely 
updated and revised as additional information is collected and 
research results are learned. Please address questions, concerns, 
and suggestions regarding these guidelines to the director of APH's 
Accessible Tests Department at 1-800-223-1839 or e-mail them to 
tests@aph.org. 

Disclaimer: Web links in this document were current as of the 
date of publication, but may have become deactivated or modified 
since then. These links are for informational purposes only and do 
not constitute an endorsement or approval of policy, views, 
products, or services of the publishing organization. 

Preferred Citation: 

Allman, C. (2009). Making Tests Accessible for Students with Visual 
Impairments: A Guide for Test Publishers, Test Developers, 
and State Assessment Personnel. (4 th edition.) Louisville, KY: 
American Printing House for the Blind. Available from 
http://www.aph.org. 

Trademarks are of their respective companies. 


American Printing House for the Blind 
1839 Frankfort Avenue 
P.O. Box 6085 
Louisville, KY 40206-0085 
Toll Free: 800-223-1839 
Fax: 502-899-2219 
www.aph.org 
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INTRODUCTION 
Purpose of Document 


The American Printing House for the Blind (APH) is committed to ensuring 
that educational materials are accessible to students with visual 
impairments. Students with visual impairments include those with some 
usable vision, as well as students with no usable vision. This document is 
provided as a guide for making tests accessible in tactile, large print, and 
audio formats. It is anticipated that this guide will be used as a tool for 
implementing appropriate guidelines as test publishers, test developers, test 
editors, and state assessment personnel are developing and adapting tests 
and assessments. Prior planning using the contents of this document will 
help ensure that tests are accessible and will reduce the need to retrofit a 
test. Questions concerning this document, the specific guidelines, or re¬ 
sources discussed can be addressed to APH's Accessible Tests Department at 
1-800-223-1839 ortests@aph.org. 

Federal and State Mandates 

Federal and many state laws require that all students be assessed through 
state assessment procedures using the appropriate accommodations. In the 
school year 2005-2006, states were required to assess all students' 
progress annually in mathematics and reading in grades 3-8 and once in 
grades 9-12. In addition, by 2007-2008, states were required to assess all 
students' progress in science, at least once in grades 3-5, 6-9, and 10-12. 
Assessment results of all students must be reported publicly, and these 
results are to be incorporated into the state's accountability plan. 

Satisfying these federal requirements necessitates careful attention to 
making tests accessible for all students. Students with visual impairments 
have some unique communication needs that must be addressed as tests 
and assessments are made accessible for them. Converting test items into 
braille, tactile graphics, large print, or audio format fails to guarantee that 
the items are accessible. For example, test items that instruct the student 
to "draw the results of the following" or "write a story based on the 
picture" are not truly accessible to braille readers. Other examples of this 
misconception are discussed within each media-specific section of this 
document. Careful planning during test development can help ensure that 
tests are accessible, while maintaining the rigor intended. 
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Expectations for Students with Visual Impairments 


If students with visual impairments are to participate effectively in state and 
national testing programs, they must have opportunities to learn academic 
skills that will be assessed. These opportunities often are overshadowed by 
special skills training to such students, who may miss all or part of academic 
classes in order to obtain the skills essential for using braille, assistive 
technology, and/or independent living skills, including orientation and 
mobility. 

In addition to providing the training of special skills, school personnel must 
be aware of each student's need for instruction in all academic areas. This 
may entail extended days or school years or supplemental instruction by 
other agencies that serve students with visual impairments. 

Students with visual impairments must spend their educational time 
working toward academic content standards, learning special skills needed 
for independent living, exploring appropriate media for access to printed 
material, and evaluating ways of communication that are effective for them. 
Access to printed material may include braille, tactile graphics, regular print 
with magnification devices, large print, the use of a human reader, auditory 
access, or technology access that provides braille, print, or auditory infor¬ 
mation. No single method will work for every student, in all situations. 
Educational personnel must ensure that students are exposed to and have 
opportunities to try all options of access. A student's communication mode 
must be based on what works for him or her. See Appendix A for a 
discussion of braille versus auditory access. 

School personnel must maintain high expectations for the education of 
students with visual impairments. If opportunities to learn are present in 
the curricula, students will have the experiences needed to learn difficult 
skills such as map and graph reading, production of graphs and charts, 
reading technical materials, or computation of advanced mathematics. 
Students cannot be denied their right to learn difficult skills just because 
they have a visual impairment. It is these more difficult skills that are being 
assessed routinely on most state and national assessments of student 
progress in reading, mathematics, and science. 
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Universal Design 

In the construction and administration of tests, the process of universal 
design helps to ensure accessibility for a multitude of students. Universal 
design provides the widest range of students the ability to demonstrate 
adequately their skills and knowledge. This process should retain the 
validity of inferences drawn from the test results. 

The concepts of universal design apply to instruction as well as assessment. 
During instruction, universal design enables investigating appropriate 
methods, practicing skills and knowledge using appropriate methods, 
experiencing trial and error to determine the best methods, and 
discovering the success of knowledge and skills learned using the best 
methods for each individual student. During assessment, universal design 
becomes the process of ensuring that the majority of students can 
demonstrate their knowledge and skills. Both aspects of learning, 
instruction and assessment, are driven by the standards of each individual 
state. 

To ensure that an assessment system is fair and accessible to all students, 
states are required to document how they include the principles of universal 
design in the item review process. Generally the principles of universal 
design include (Thompson &Thurlow, 2002, Thompson, Johnstone, & 
Thurlow, 2002): 

• Attention to an inclusive assessment population, 

• Constructs, including content and cognitive complexity, that are 
precisely defined either through states' standards or the test item 
specifications, 

• Accessible test items, as determined by item writers and review teams 
that include personnel familiar with various media (braille, tactile 
graphics, large print, regular print, and audio), 

• Non-biased test items, as determined by item writers and review 
teams, 

• Test formats, response options, and scoring policies that are amenable 
to various approved accommodations needed by students, 

• Simple, clear, and intuitive instructions and procedures, 

• Comprehensive and relevant language that provide needed distracters 
in test item foils but are not designed to confuse the student, and 

• Maximum legibility of print formats including formats that are free from 
clutter and void of grayscale. 
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Suggestions for applying universal design to item development include: 

• Ensuring that test item writers are trained in concepts of universal 
design, 

• Providing test item writers and reviewers with construct and construct- 
related information during the construction and review of test items, 

• Examining each test item for universal design principles(linguistic 
complexity; cognitive complexity; formatting; bias issues; modalities 
of braille, large print, and audio; and response formats to be allowed), 

• Recommending allowable accommodations for test administration, 

• Re-examining all test items for fidelity to the construct, and 

• Field testing all test items with intended populations. 

The principles of universally designed assessments are the basis for many of the 
guidelines provided in this document. Additionally, the references listed at the 
end of this document have been written by individuals involved in ensuring the 
accessibility of materials for students with visual impairments. This document 
describes guidelines that support braille, tactile graphics, large print, and audio 
production of test items. 
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GENERAL GUIDELINES FOR 
ACCESSIBLE TEST FORMATS 

Students with visual impairments may require testing materials in regular 
print, large print, braille, tactile graphics, audio formats, or some 
combination of these formats. The provision of a test and related materials 
in braille, large print, or audio provided an individual student should be 
based on the medium used by the student, as identified in the 
Individualized Education Program (IEP) document. Alternate format tests 
should be used only by students who use that medium to access printed 
textbooks and other instructional materials. 

Students with visual impairments can be, and must be, made part of the 
state's assessment program through use of accommodations that allow 
them to demonstrate their knowledge and skill acquisition, as outlined in 
each state's standards and assessment system specifications. Regardless of 
the media chosen, students may need access to special materials such as 
braille paper, bold line writing paper, talking calculators, abacuses, raised or 
bold line rulers, braillewriters, slates and styluses, word processors, or other 
materials and devices. A more thorough discussion of accommodations is 
provided in the section on Accommodations in Testing Students with Visual 
Impairments and in Appendix F. 

The following general guidelines are recommended for all formats that are 
developed for accommodating students with visual impairments. Various 
aspects of test construction and implementation are addressed in this 
section. 


Contract Development 

1. Contracts between states and test publishers/producers must include 
provisions for state approved alternate media (braille, large print, 
audio editions of tests, and scripts for oral presentation of tests) 
including answer sheets and practice tests. 

2. Test publishers need to have the capability of providing the test 
administration manual in braille, large print, or audio for test 
administrators who are visually impaired and need accessible media. 
The contract should state if test administration manuals are needed in 
accessible media. 

3. Contracts must include timelines for development, proofreading, 
revising, and production of braille, tactile graphics, large print, and 
audio test formats and accompanying practice materials. 
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4. Contracts regarding accessible media should guarantee that each 
medium of test materials and practice materials is produced by the 
same entity to ensure consistency in format and graphic production 
techniques. Every effort should be made to ensure consistency of 
presentation from one year to the next, and from one level of the 
test to the next. 

5. Contracts may need to include specifications on tools and materials 
that need to be developed or provided to test takers using alternate 
media, e.g. a braille ruler, a tactile or large print protractor, or 
periodic table of the elements, real money for money related test 
items, or some actual objects such as a ball or cube. 

6. Contracts may need to include plans to ship special versions of tests 
separately from regular print versions so that distribution of the 
accessible formats occurs in a timely manner. 

Test Development 

1. Test development must ensure that test score inferences reflect 
intended constructs and not disability characteristics (AERA, 

2000 ). 

2. The construct to be measured must be specified in documents and 
made available to test item writers and reviewers and to accessible 
media producers. 

3. Availability of item specifications is essential in determining 
appropriate accommodation use and in the reproduction of test 
items to be presented in braille, tactile, large print, or audio 
formats. 

4. Test publishers must maintain access to experts in the media of 
braille, tactile graphics, large print, and audio, who can provide 
information concerning test development and transcription and 
tactile graphic design, and who are able to proofread test materials 
before mass duplication, and otherwise ensure that materials are 
provided in a timely and accurate manner. Proofreading the braille, 
large print, or audio version of the test before multiple copies are 
made confirms that the material is readable and that the adapted 
test follows the print copy in numbering and lettering of test items 
and answer choices, and that the graphics are readable and located 
correctly. The proofreader must also check for proper formatting. 

5. Validity issues concerning all accessible formats and accommodation 
needs should be discussed during test development (Phillips, 1994). 
The provision of a test in accessible media should be considered a 
valid accommodation as long as it retains the construct that the test 
was designed to measure. If a performance item requires drawing, 
consider allowing an explanation or description as a valid response 
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option. If such a revision is allowed, scoring criteria must include 
information on this option. 

6. All directions on a test should be worded to allow for alternate 
response methods. For example, use of directions like "circle the 
answer" should be replaced with "indicate or mark the answer." 

7. Specific guidelines on any test format changes, allowable 
accommodations (including time allowances), and general 
assistance that can be provided to the student must be stated in 
the test administration manual or supplemental administration 
materials. 

8. Test item development and review committees should be made aware 
of alternate media issues regarding the use of either complicated or 
nonessential pictures and graphics. 

Item Development and Review 

1. Educators with specialization in the field of visual impairments 
must be included in the test item development process. 

2. All test items must be reviewed by persons familiar with visual 
disability issues to ensure that no test item is biased or 
discriminatory toward persons with visual impairments. 

3. It is recommended that as much information as possible be 
included in the text of a test item. This will help prevent the 
introduction of pictures that contain information necessary for 
selection of the correct answer, but which cannot be adequately 
brailled, presented in large print or tactile graphics, or described in 
audio format. 

4. In general, use of "vision specific" language can be maintained, 
e.g., "Look at the following list of animals." 

5. The test item pool must be large enough for bias and item review 
committees to replace items determined to be biased or inaccessible 
in braille, large print or audio formats, or tactile graphics. 

6. A representative sample of students with visual impairments 
must be included in any field-testing of the assessment, as 
prescribed in Standard 10.3 (p. 106) of the Standards for 
Educational and Psychological Testing (1999). 

7. All practice materials must be provided in accessible format at the 
same time that print practice materials are provided. Allow 
sufficient time for accessible format preparation. 

8. Provisions should be made to conduct item analyses for accessible 
format test items. 
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Accessible Test Development 

1. To ensure that quality materials are developed, state assessment 
programs should contract with an agency or persons experienced 

in producing braille, tactile graphics, large print, and audio formats. 

If a multimedia presentation is to be used by test takers, it is 
important that the accessible media producer(s) coordinate pre¬ 
sentation of the test items between each of the media. 

2. Production of the alternate format test includes the editing, tran¬ 
scription, reformatting, design, and proofreading of the 
alternate media. 

3. Holding a conference call with all parties involved before the 
accessible media producer begins to review/ edit the test items helps 
to maximize a successful experience and end product. 

4. The name and phone number of the customer's primary contact 
person needs to be provided to the producer of accessible media 
to facilitate timely production. 

5. Accessible format producers will need access to a primary contact 
person, as well as item specifications that include information 
about the skill and construct being assessed. 

6. Test items should be deleted or substituted only if the item cannot 
be provided in braille, tactile graphics, large print, or audio format 
without significantly changing the item and the intent of the 
question. Although not recommended, some test items may need 
to be omitted if they are not adaptable as determined and advised 
by item reviewers with expertise in the format under consideration. 
The deletion or substitution of items should happen infrequently, 
particularly if educators with specialization in visual impairments 
have been involved in the item development process. Attention to 
universal design during test development will also reduce the 
probability that a test item will have to be deleted. 

7. If items are omitted in alternate versions, the test score 

must be rescaled so that braille, large print, and audio format users 
are not unfairly penalized and so that scores can be obtained for 
diagnostic and accountability use. The original numbering system 
should be maintained and the word "omitted" inserted in place of any 
item that had to be omitted. 

8. Responses from the primary contact person regarding questions and 
requests for substitutions require a quick turn-around time in order to 
ensure accuracy and timeliness of delivery of accessible media. 

9. Substituted items should assess the same skill and have equal 
value and validity. Substituted items must maintain the correct 
answer in the same position as that of the original test item. 
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10. All field test items and sample questions must be included in 
accessible format test versions. 

11. Test contracts must indicate preferred publication strategies, such 
as brailling on both sides of the braille paper (referred to as 
interpoint braille), preferred methods of producing tactile graphics, 
and binding of the braille test materials. Assistance with determining 
these specifications is available from APH. 

12. Braille tests are generally produced using contracted braille, the 
typical method for producing braille in which short forms of words are 
used. If the test is for a young child, a new braille reader, or 
someone struggling to learn braille, a test may be needed in 
uncontracted braille, whereby every letter of every word is 
represented by an individual braille cell. 

13. The format of an accessible media test edition must follow the 
print format as much as possible. That is, ideally the number of 
test items and test sections should match that of the print format, 
as should the order of the test items and test sections. Deviations 
from the print version of the test must be outlined in a print copy of 
Test Administration Notes for the altered format. Test Administration 
Notes must include reference to print versions with associated 
accessible format page numbers, identify passages and items by page 
(print and alternate format), and provide indication of any changes 
made to the alternate format. Appendices B and C contain templates 
for creating Test Administration Notes for braille and large print 
formats. 

14. Special requirements, such as an independent proofreading of test 
materials, exact print reproductions of the braille/tactile test items, 
or any print labels to be included on braille or tactile graphics need 
to be considered and included in the contract. 

15. APH's policy in accessible test production includes close collabora¬ 
tion with, and approval from, test publishers and content 
specialists to ensure that edited items are acceptable as edited. 

16. Test security and confidentiality standards must be upheld during 
the process of developing accessible formats. 

Test Administration 

1. Computers and adaptive technology, electronic note takers, 
cassette player/recorders, the cassettes, CDs, etc., must be 
inspected for proper functioning prior to their use during a test. 

The test administrator or proctor should be instructed on how to 
proceed if equipment fails or malfunctions during administration of 
the test. 
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2. Each test administrator or proctor of a student using an alternate 
medium test or a combination of media should be assigned a 
testing packet that includes a list of materials needed (approved 
technology or other manipulatives, such as a talking calculator, 
braille or large print ruler, braille paper, bold line writing paper, 
raised line graph paper, etc.) 

3. The test administrator or proctor must ensure that special 
tools and materials noted on the student's IEP and used for 
instructional purposes as accommodations are available, as 
needed, to students in the test-taking environment. For example, 
if a visually impaired student routinely uses an abacus in the 
classroom when sighted students are allowed to use a pencil and 
paper for computational purposes, then an abacus must be 
available during a test. See Appendix D for further explanation on 
the use of an abacus in test-taking situations. Specialized tools 
and materials should not be provided if their use presents an 
unfair advantage. 

4. In preparation for test administration, the test administrator needs 
to review the original test(s), the alternate format/s of the test/s, 
the original test administration manual(s), the test administration 
manual/s for accessible media, and the test administration notes 
for the special format/s. These materials should be provided to the 
test administrator under secure and confidential means two full days 
prior to test administration. This time is needed so the test 
administrator can plan appropriately for administration of the test(s) 
in alternate media. 

5. Prior to testing, the test administrator or teacher must ensure 

that the test is available in a student's primary or preferred reading 
medium or combination of media, and that the student has sufficient 
proficiency in use of this medium and related tools such as 
computers, assistive devices, CD players, or braillewriters. 

6. If students are expected to bring select tools and materials to the test 
environment, they need to be notified of this ahead of time. 
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BRAILLE AND TACTILE GRAPHICS 

The information in this section describes methods for developing and 
implementing assessments for students with visual impairments who 
require braille text or tactile graphics. While some technology provides 
auditory access to print, braille is critical to literacy and must be an 
option for those students who routinely use it. See Appendix A for a 
discussion of braille versus auditory access. 

Generally, learning to read braille is no more difficult than learning to 
read print. The tactile process is different from the visual process and 
creates the following considerations: 

• Braille (tactile reading) consumes more time than does visual 
reading, as students who read braille typically read at fewer words 
per minute than do students who read print (Trent &Truan, 1997), 
and 

• Braille reading requires tactile training in page orientation and 
reading and interpretation of tactile graphics. 

Designing Tactile Graphics 

The following are aspects of test items that need special consideration 
when reviewing and designing for production as tactile graphics: 

• Complicated graphics that contain multiple layers or pages of 
information 

• Three-dimensional objects from a particular visual perspective, e.g., 
a top view of a house or pyramid 

• Rotation items that use letters of the alphabet (print letters 
rotated or flipped) 

• Science items that use pictures to demonstrate 
experiments and other scientific concepts or processes (cell, 
digestive or muscle systems, etc.) 

• Map reading items that depend on visually recognizable and 
unlabeled continents, countries, or states, e.g., Africa, Italy, or 
Florida 

• Visual recognition items (interpreting a picture without supporting 
text) 

• Items that require interpretation of complicated drawings, e.g., 
cross-sections of diagrams 

• Optical illusions 
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These types of items frequently require extensive revision during the 
production process. For example, a text-based description in addition to 
the tactile graphic may be needed. The accessible media producer may 
request a test publisher to substitute such items with those that can be 
made more accessible and which will retain similar, if not identical, 
concepts and have the same weighted score. 

Braille Translating (Transcription) Process 

Consideration of the following points will facilitate the production of test 
materials in braille format for students with visual impairments: 

1. Test developers and publishers must ensure that contracts for braille 
materials specify the use of braille transcribers who are certified by 
the National Library Service (NLS), experienced at transcribing tests, 
and knowledgeable of braille formats. Braille formats must be 
modeled after those of the Braille Authority of North America (BANA) 
Guidelines, found in Braille Formats: Principles of Print to Braille 
Transcription , 1997. 

2. As a test is edited for braille transcription, necessary changes will 
be made to make the material accessible to braille readers. 

Correct braille transcription also requires that BANA specifications 

be observed. Simplification and/or labeling of some graphic material 
will likely be necessary. Simplification entails the elimination of some 
artistic features, removal of some superfluous material (without 
eliminating distracters and other text material that is necessary to 
maintain the validity of the test item), or movement on the braille 
page of some text or graphic components for more efficient 
readability by the braille reader (moving a scale, legend, or compass 
rose on a map to a different location). Even simple tactile graphics 
can be very difficult to interpret; some additional labeling may be 
needed for the test taker to read and understand the tactile graphic. 
Note that simplification and labeling are done relative to the 
construct being tested. If during the test editing process, it is not 
clear what is being tested, the test publisher will be consulted for 
clarification. 

3. Reproduced references, such as tables of content, dictionary pages or 
indices, may need to be shortened in the braille test version while 
maintaining correct answer choices and foils. This is done to contain 
the braille item to one page, if possible. 

4. Provision of open-ended items in braille format must indicate to 
the braille reader the amount of space provided for the answer. 
Directions must specify the space provided by suggesting the time 
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needed to complete the item or by indicating the approximate 
page area or the number of lines or paragraphs. Generally, one page 
of print is equal to about two pages of braille unless graphics are 
involved which will add to the page length. Directions may indicate 
that there are four print lines or eight braille lines available for 
responding. 

5. Unnecessary boxes and framing of material may be omitted 
unless the framing provides a separation of graphic material from 
text or encloses a group of scattered or randomly placed objects. 

6. Specific braille codes exist for transcribing literary works, 
mathematics, and science materials into braille. When brailling the 
content of the print version of the test, braille transcribers must 
follow the standards of code for braille transcription. These codes 
are provided in English Braille American Edition, 1994; The Nemeth 
Braille Code for Mathematics and Science Notation , 1972 Revision; 
and Braille Formats: Principles of Print to Braille Transcription , 1997. 
All three manuals are available from the American Printing House for 
the Blind. 

7. An experienced braille proofreader must be utilized for proofreading 
all materials and, in particular, examining all tactile graphics to 
ensure readability and accuracy. 

8. Experienced braille readers might also need to transcribe students' 
braille responses into print for scoring. See the section on Guidelines 
for Braille and Large Print Test Response Transcription. 

9. Braille versions of a test may include transcriber's notes (notes to the 
braille reader from the braille transcriber about the use of special 
symbols, and use of any special formats). Transcriber's notes must 
be written in print within the Test Administration Notes for Braille 
Edition (Appendix B). The number of transcriber's notes in tests 
should be kept to a minimum. 

10. Test security and confidentiality standards must be upheld by braille 
test transcribers, tactile graphic designers, and proofreaders. This 
includes the following: 

• Keeping testing materials in a secure place to inhibit access by 
unauthorized persons, 

• Not sharing information or implying content contained in the 
testing materials with other persons, 

• Maintaining discretion about the work being performed, 

• Returning all materials to the contracting source, and 

• Maintaining confidentiality of test content. 
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Tactile Graphics 

This section offers information regarding the use of tactile graphics when 
testing students with visual impairments. Graphic material, which 
includes maps, charts, graphs, diagrams, and illustrations, frequently 
contains information that is difficult to present in a tactile format. 
Research supports the use of tactile graphics and "the idea that visual 
experience and visual imagery are not required for the perception of 
simple tangible pictures . . ." (Heller, et al., 2002, p. 349). It is possible 
to provide many types of graphic material in braille or raised line 
drawings. However, certain types of graphic materials either cannot be 
provided in braille or tactile formats, or they are so complex that doing 
so produces a graphic that cannot be read and interpreted by the test 
taker. 

Most maps, charts, graphs, and diagrams can be translated into tactile 
form if the test publisher will allow some editing. Editing could involve 
eliminating the shading used solely for visual effect, reducing the number 
of distracters, providing two or three charts to present the same 
information as one complex print chart, using text based descriptions to 
supplement or replace graphics, or using symbols and words with a key 
to provide information. Edits needed to convert print graphics to tactile 
graphics need to be approved by test developers or publishers. 

Most print materials use graphics to emphasize a point, provide another 
format for information, or provide visual appeal. Because graphics are 
common in text, training in reading graphic material and interpreting a 
written description of a graphic are important skills for the student with 
a visual impairment to learn. Guidelines for tactile graphic materials are 
described on the next few pages in terms of general guidelines, design, 
symbols, lead lines, labels, and indicators and scale. 

General Guidelines: 

1. Graphics in mathematics tests must follow provisions of The 
Nemeth Braille Code for Mathematics and Science Notation , 1972 
Revision , BANA (1983). 

2. Decide if a tactile graphic is needed. Omit the graphic if it is 
purely decorative. Consider using a text based description to 
either supplement or to replace all or part of a graphic. 

3. Graphics should be tactually clear and contain only relevant 
information based on an understanding of what is being taught 
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and what the student's task is. Visual information that is 
irrelevant to the meaning or purpose should be omitted. 

4. Graphic material should be simplified without omitting needed 
information or creating an unfair advantage by alluding to the 
answer. 

5. Picture descriptions should be presented concisely within the 
student's test booklet if information in the picture is vital to 
answering one or more test items. 

6. Picture descriptions will appear as needed in transcriber's notes 
in appropriate places throughout the test and must be included in 
the Test Administration Notes. 

7. Some graphics are best handled by supplementing the image 
with a heading, label, description, or key. Edits must be made 
carefully so that the braille reader is not unintentionally given an 
advantage or cue to the correct response. 

8. Consider splitting complex graphics into separate drawings 
showing layers of information, unless this adds complication for 
the test taker. 

9. In general, use texture to add information and draw attention to 
select parts of a tactile graphic. 

10. When necessary, to avoid confusion and accentuate important 
information, use different areal symbols (texture) to differentiate 
between bodies of water and land on maps. 

11. Charts and graphs should be confined to one page when possible. 
If graphics and the accompanying test item require more than 
one page, use facing pages to present graphics and the 
accompanying test item if possible. 

12. If a braille test taker is asked to produce a graphic as part of the 
test item, such a task can be achieved through the use of tactile 
graphic materials that are familiar to the braille reader. Another 
option that may be acceptable to test developers is for the 
student to describe or explain data or other information. This 
option must be approved by the test contractors and included in 
the scoring criteria. The test administrator and the braille 
reader's teacher, using the braille reader's current IEP, must 
collaborate prior to the administration of the test to ensure that 
appropriate materials are provided. For the purpose of scoring, 
student-produced graphics will need to be hand-scored or 
transcribed into a print graphic by persons familiar with braille, 
braille readers, and the content area being tested. See section 
on Guidelines for Braille and Large Print Test Response 
Transcription. 
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13. An experienced braille reader must proofread all tactile graphics 
prior to mass production of the braille test to ensure readability 
and accuracy. 

(Kapperman, G., Heinze, T. &Sticken, 1, 2000; Poppe, K. &Otto., F. 2002; Ross, D. 

B. & Robinson, M. C., 2000; Spence, D. & Osterhaus, S., 2000) 

Design: 

1. Avoid clutter and simplify. 

2. "Clutter" occurs when different symbols and lines are so close or so 
similar that they become hard to distinguish. Spacing is the key to 
avoiding clutter. 

3. Symbols and lines closer than Va" may be difficult to differentiate, 
depending on the medium and tools being used. 

4. Shapes with sides less than Vi" long may not be recognizable. 

5. Use different textures for lines so that test takers know which part of 
a line to follow when two or more line segments cross or meet. 

6. "Simplify" means to eliminate unnecessary elements of the original 
picture. Focus on the relevant parts and omit details that are purely 
decorative or distracting. 

7. When the print picture includes people, animals, objects, etc., replace 
them with simple lines, symbols, and/or labels (e.g., use the label 
"hand" instead of drawing a hand or use a triangle instead of a cat or 
dog). 

Symbols (Lines, Points, and Textures): 

1. Limit the lines, points, and symbols on a drawing to those that can be 
easily differentiated by touch. Use the most prominent symbols for 
the most important features in the graphic. Avoid high or "noisy" 
texture, which draws attention away from the key features. 

2. Be consistent in using symbols within graphics of the same type 
within the same transcription (e.g., always use the same symbol for 
water on maps). 

3. Use different tactile symbols for different types of information (e.g., 
in a map of the United States, the tactile line used to indicate state 
borders should differ from the tactile line used to indicate 
international borders). 

4. Lines, points, and braille must be physically separated by at least 
1 / 8 ". 

a. This distance may need to be extended to at least Va inch depending 
on the medium and symbols used. 
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b. Apply the 1/8 inch separation rule to all features that are separate, 
even if doing so introduces some spatial distortion. 

Lead Lines: 

1. Use lead lines only as needed; options for lead lines include use of 
keys or notes. 

2. Avoid using arrows as lead lines. 

3. The linear symbol used for lead lines should be different from any 
other lines used in the graphic and should be tactually distinctive but 
less prominent, such as a low relief raised line. 

a. A lead line should begin as close as possible, without causing 
interference, to either the first or the last letter in the label, and it 
should end as close as possible to the feature being labeled. 

b. In general, a lead line should not cross over another line. When this 
is unavoidable, it may make the graphic more readable by breaking 
the lines of the graphic to allow the lead line to "pass through." 

Labels and Keys: 

1. Explain and define all graphic symbols. 

2. Identify all important features (e.g., capitals, bodies of water, etc.) of 
the graphic, even when not labeled in the print version. Place titles 
at the top of the page. Avoid making unlabeled graphics. Exceptions 
may exist in some testing situations. 

3. Position labels closely to the objects to be identified to better ensure 
recognition. Single letters on the graphic should be preceded by 
either the letter sign or the capital sign. 

4. Use two-letter U.S. postal codes where applicable (and other two- 
letter codes where postal codes are not applicable) for labeling state 
names on maps. 

5. Words in labels need not be capitalized if their meaning will not be 
confused and rules of punctuation are not violated. 

6. Place all abbreviations in a key and place the key above the tactile 
graphic. 

7. Present all braille labeling within tactile graphics horizontally. 

Indicator and Scale: 

1. Graphics depicting measurements must maintain accurate and true 
proportions to match the answer choices. If answer choices must be 
changed, the correct response must be located in the same position 
as the original correct response option. 
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2. Position scale and other indicators as consistently as possible, 
preferably above the accompanying tactile graphic. 

3. When it is necessary to change the scale, this fact may need to be 
indicated in a transcriber's note. 
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LARGE PRINT TEST FORMATS AND GRAPHICS 

Some students with visual impairments read regular print materials and 
enlarge the print, as needed, by using optical devices. Some read large 
print materials. This section offers information regarding the develop¬ 
ment and implementation of assessments for students with visual 
impairments who require large print materials. Generally, two popular 
methods exist for enlarging tests. The regular print test can be enlarged 
through photocopying, or an electronic version of the test can be 
manipulated to reformat test items and enhance the readability of text 
and graphic as needed. The latter method is preferable unless issues 
outlined in this section have been addressed during the test development 
and the regular print test has been designed using universal design 
principles. Manipulating an electronic version of the test can best yield a 
large print version that incorporates the optimum reading mode for the 
student who uses large print. 

Generally, reading skills that are difficult for a person with low vision who 
reads print include the following: 

• Reading at a speed commensurate with regular print readers, 

• Reading for extended periods of time, 

• Visual scanning and skimming of text, 

• Shifting gaze from a picture or graph to test item and back 
again, 

• Shifting gaze from test booklet to answer sheet documents, 

• Visually capturing an entire picture, 

• Moving from one line of text to the next, 

• Locating pictures and text presented in random locations on 
page, 

• Interpreting pictures (particularly complex pictures), 

• Differentiating between subtle colors and patterns used in 
pictures or graphs, and 

• Filling in answer choices on regular print answer documents. 

Consideration of these points, particularly in relation to universal design 
of test format and printed text, will facilitate the production of test 
materials in large print format. As well, most of these guidelines are 
applicable to regular print tests that may be used by students with low 
vision. Information provided on font, spacing, shading and contrast, 
pagination, and test booklets is a summary of work done by Elaine 
Kitchel, presented as "Reading, Typography, and Low Vision," a PowerPoint 
presentation (APH, 2002). Research completed by G. E. Legge et al., 
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(Reported in "Psychophysics of Reading" 1985 through 2002 in Vision 

Research) supports the guidelines listed in the following section. 

Test Format 

1. Large print versions of a test and test practice materials should be 
reformatted from the regular print version so that adaptations can be 
made to font style, print size (point size), line length spacing, 
shading, graphics, and the number of items on a page. 

2. Text should consistently begin at the top left-hand side of a page. 
Titles of pictures or graphs should appear at the top of the graphic. 

3. Labeling should be presented horizontally rather than vertically as a 
general rule. Exceptions may be labeling of y-axes on graphs, etc. 

4. Items that typically present the most difficulty during conversion to 
large print format include the following: 

• Complicated, multi-shaded drawings with extensive details, 

• Grayscale drawings that provide little contrast, 

• Colors that cannot be differentiated by persons with color 
blindness, 

• Large maps that cannot be contained on one page if enlarged, 

• Extensive charts with multiple columns, and 

• Charts and graphs that extend over several pages. 

5. If testing materials are enlarged merely through photocopying 
(not recommended), the font size will vary depending on the original 
print font. When tests are enlarged, the font size of all text, 
including labels on graphs, rarely meets the 18-point size required. 
Enlarged materials must be reviewed and proofread before mass 
copying or distribution to ensure that print and background contrast 
are adequate, that pictures and graphs are readable and complete on 
the page, and that items assessing measurement are accurate and 
have viable answer choices. 

Fonts 

1. Print measuring 18 points or larger is considered large print. Point 
sizes between 12 and 16 points are considered enlarged print. 

2. Occasionally a test in a print size larger than 18 point will be 
requested. In such cases, the publisher must determine if material 
can be adequately presented in a larger point size. 

3. Decisions about the size of print and font style must be made by 
the test publisher and discussed with a person who has knowledge 
of large print use and the intended test takers. 
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4. Font styles that are decorative or cursive should not be used. 
Standard sans serif fonts with easily recognizable characters are 
recommended. Verdana, APHont, Arial, Antique Olive, and Helvetica 
are reliable choices for readability. Note: APHont, a font for people 
with low vision, developed by the American Printing House for the 
Blind (APH), embodies characteristics needed by low vision readers as 
identified by research. A free version of APHont is available from APH 
at http://www.aDh.org/products/aphont qet.html . 

5. Large print should have x-heights (distance from the top to 
bottom of a lower case "x") and t-heights (distance from the bottom 
of the "t" to the cross bar of the "t") of at least 1/8" with a 
thickness of 2 points. Eighteen point Verdana, APHont, Antique 
Olive and Helvetica meet this standard. 

6. The use of bold print, underlined print, or quotation marks for 
highlighting text is preferable to using italics. Italics should only be 
used when absolutely necessary. Sample test items, if provided, 
should be presented in the same font size and style as that used for 
the actual test items. Letters incorporated into math problems, e.g., 
letters within algebraic equations, are also more readable when 
displayed in a non-italic, sans serif font. 

7. Headings and subheadings (captions, titles of diagrams and charts) 
should be larger and bolder than other print and set in a font style 
that differs from that of the general text. Acceptable typefaces for 
this use include Arial Black, Helvetica Black Bold, Lucida Sans Bold, 

Era Bold ITC, Verdana Bold, Antique Olive Bold, and Helvetica Bold. 

8. All text, including labels and captions on graphs, pictures, diagrams, 
maps, charts, equations, exponential numbers and letters, subscripts 
and superscripts, notes, and footnotes, must be presented in at least 
18-point type, in order to meet the APH definition of large print 
(Kitchel, 2001). 

Spacing 

1. Leading or spacing between lines should be at least 1V4 spaces to 
allow persons with low vision to effectively move from line to line 
in the text. 

2. Block style formatting and 1" margins are recommended. 

3. Format should include justification of left margins, and unjustified 
right margins (rag right) for ease in reading and transferring from 
line to line. Avoid the indentation of paragraphs. 

4. Avoid dividing words between lines. 

5. Columns of text, excluding graphic material, should be at least 39 
characters in line length. Generally, for efficient reading, columns 
should be avoided. 
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6. Test items and accompanying diagrams, pictures, and graphics 
should be located close to each other and on the same page if 
spacing permits. If this is impossible, test questions, diagrams, and 
answer choices should be placed on facing pages or follow closely so 
that page turning is reduced to a minimum. 

7. Research indicates that readers with low vision and readers with 
normal vision read a wide-bodied font faster and with better 
comprehension than they read a variable-spaced font (Mansfield, & 
Legge, & Bane 1996). 

Shading and Contrast 

1. Grayscale and shading should be avoided, particularly when 
information needed for answering a test item is provided. 

2. The highest possible contrast should be used for text and back¬ 
ground, with attention to the use of color. Certain color combi¬ 
nations other than black and white may be unreadable to 
persons with low vision or persons with color blindness. A good 
rule of thumb on use of colors is to use colors that are far apart on 
the color wheel and avoid using colors that have similar saturation 
(color depth). Blue and yellow, for example, provide a high degree 
of contrast when used together. Red and green should be avoided 
because they are the most troublesome colors for persons with color 
blindness. 

3. Large print must not be used over a background design or other 
graphic material. 

4. Glossy paper may cause unnecessary glare. Dull finish paper in white, 
ivory, cream, or yellow is recommended and best complemented 
with black print. 

5. Unnecessary boxes and framing of material should be omitted unless 
the framing provides a separation of graphic material from text or 
encloses a group of scattered items. 


Cautions for Use of Recycled Paper 

Whether recycled paper is appropriate or not for use by individuals 
with low vision depends on its color and its thickness. The color 
cannot tend toward gray, blue, or green. If it is slightly gray, blue, or 
green (and many recycled papers are) it can substantially reduce 
contrast. What seems like a minor contrast difference to a sighted 
person can be a big contrast difference to a person with low vision. 
However, if the tint of the paper tends toward beige, peach, pink, or 
yellow then it would be fine. In addition, there should be no speckles 
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in the paper. Best color choices for recycled paper would be cream, 
beige, or white. Finally, the paper should be thick enough to prevent 
bleed-through of inks. The paper needs to be thick enough to allow 
printing on both sides of the sheet with no bleed-through. The same 
cautions apply to materials for persons with color blindness or color 
vision deficiencies (Kitchel, 2009) (see section on color vision issues 
below). 


Pagination 

1. Repagination of original test materials is preferable to increasing the 
overall page size. 

2. While double-sided pages are generally preferable, avoid double 
sided copying if print will "bleed" or show through or otherwise 
obstruct clear reading. 

3. Where blank pages must appear, type the words "Blank Page" 
near the top left hand side of the page. 

Format of Test Booklets 

1. Depending on test length, large print copies may need to be 
separated into several booklets. 

2. Generally, the test booklet should be no larger than 9" x 12", 
particularly for young students as well as other students with various 
physical conditions. 

3. The binding of the large print booklet(s) should allow each page to 
lie completely flat for whole page viewing and ease of handling. 

Large Print Graphics 

The following guidelines provide information concerning the use of 
graphics in testing students with visual impairments who use large print 
formats. Work by the Large Print Atlas Focus Group (2001), who met at 
the American Printing House for the Blind, is included in this discussion. 

The complexity of some graphic materials prohibits their being provided 
in large print unless they are modified to become more readable when 
enlarged. Most maps, charts, graphs, and diagrams can be enlarged if 
the test publisher agrees to some editing. Editing could involve the 
elimination of shading, the reduction of some distracters, the insertion of 
a key, or the separation of one chart into two or three. 

Guidelines for large print graphics include the following: 
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1. Graphics in large print must exhibit good contrast, clarity, and 
accurate details and information. 

2. No test item should rely solely on a picture for information 
needed to answer the test item. In consideration of universal 
design, include a text description of every non-text item. 

3. Generally, pictures should be retained in the large print format. 
Editing for shaded material and clarity may be necessary. Some 
pictures that would need extensive editing and provide little or no 
cues for the large print reader may be considered for elimination. 
Purely decorative graphics should be deleted. 

4. Overlaid print on a diagram or graph should be avoided. While 
visually pleasing in some instances, this technique is difficult for 
persons with low vision to read. 

5. Multi-color graphs that use closely related colors may conceal 
vital information from the test taker who is unable to 
distinguish between the colors. Two to three contrasting colors 
or black and white are recommended. 

6. All graphs should contain short, descriptive headings or titles. 

7. Compass points, numbers, and vital information on graphs must 
be enlarged sufficiently for the low vision reader. 

8. Map symbols must be easily distinguishable and relevant. 

9. Map legends should appear near the top left hand corner of a 
map, if possible, and include a visually distinctive border. Use 
contrasting colors and distinguishable symbols rather than 
reproducing different sizes of the same symbol. 

10. If possible, map scales, too, should be positioned near the top 
left-hand corner of the map. 

11. Labels should be arranged within the boundaries of the country or 
state borders whenever possible. 

12. Symbols used should be reasonable and meaningful 
representations, e.g., a fish for fishing. 

13. Boundaries between countries should be bolder and thicker than 
boundaries between states or provinces on a map. 

14. Pictures and graphs used in test questions requiring measurement 
must be true to the size intended in order to ensure that a 
correct answer is available. 

15. Test publishers and contractors will need to address the degree of 
accuracy that is expected for questions involving measuring or 
drawing. For example, some large print readers may not be able 
to distinguish between 7/16" and 8/16" on a ruler. If at all 
possible, specially designed measurement devices, such as large 
print rulers and protractors, should be provided for students in 
both the classroom and testing situations. 
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16. If a graph or table does not exceed one page in the original 

materials, then the large print version should be edited to fit on 
one page, if possible. Pertinent information and distracters must 
be maintained. 
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USES OF COLOR FOR SIGNAGE, GRAPHICS, 
TEXT, TESTS AND POWERPOINT 
PRESENTATIONS TO BE VIEWED BY PERSONS 
WHO ARE COLOR BLIND OR COLOR 
VISION DEFICIENT 

Introduction 

Color is critical to the conveyance of meaning in signage, graphics, text, 
PowerPoint® presentations, tests, and other written presentations. 

However, some people, specifically those with color discrimination difficulties, 
need special consideration when color planning for educational purposes. 

Virtually all color-deficient individuals have varieties of red or green 
deficiency. (Blue deficiency is rare indeed, with only about .001% of the 
population having it.) Color blindness is normally diagnosed through clinical 
testing by a licensed practitioner. 

PROBLEMATIC AREAS 

When one considers educational materials for students who are color blind or 
color deficient, some problematic areas come to mind: 

• Use of gray-on-gray bubble sheets on test answer sheets 
(scannable answer documents) 

• Maps with indistinguishable adjacent colors, such as coloring Spain 
brown and Portugal green 

• Graphs with indistinguishable adjacent colors 

• Use of text over graphic backgrounds, as when a poem or other 
text is superimposed over a photo or drawing 

• Test questions which depend upon color identification for correct 
answers 

PREPARATION OF MATERIALS FOR PERSONS 
WHO HAVE COLOR VISION DEFICIENCIES 

Color is one of the most important aspects of visual communication and can 
be employed to generate interest or to communicate ideas or feelings. Yet 
colors for an audience with members who have color discrimination problems 
should be selected carefully to avoid conveyance of unintended meaning. 

This is especially true in educational and testing materials. Many of these 
materials rely on good color perception for the interpretation of graphs, 
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charts and illustrations. Yet even the most carefully thought-out graphic may 
lead the user to an incorrect answer because of poor color selection. 

• Select colors carefully. Besides black and white, most color blind 
individuals can only see two colors, blue and caramel (golden brown). 
Red, yellow, orange, and green take on shades of caramel; purple takes 
on shades of blue when viewed by a person with colorblindness. 

• Less is more. Too many colors used thoughtlessly can confuse and 
negate the message of a graphic. Settle on four or fewer colors and stick 
with them. Black and white are counted as colors when designing 
graphics, even though they are not usually considered colors when 
talking about vision. 

• Use contrasting colors. Contrast is an important influence on the 
legibility of graphics, especially for persons with color discrimination 
problems. Substantial contrast, i.e., the use of dark values with light 
values, between the color of the foreground and the background should 
be employed. High contrast makes materials easier to read by both 
persons with colorblindness and those with typical vision. Light letters on 
a dark background or dark letters on a light background are most 
legible, but remember the actual colors of those combinations are 
important. 


CONTRASTING COLORS APPROPRIATE FOR PERSONS WITH COLOR 
PERCEPTION DIFFICULTIES 
(in order of best contrast value) 

• Use black and white. 

• Use dark blue and white. 

• Use black and bright yellow. 

• Use dark blue and bright yellow. 

• Use dark brown and white. 

• Use pale blue and black. 

• Use yellow and purple. 

Notice that yellow is recommended as a common color for graphics to be 
used by persons with poor color discrimination. This is because yellow 
maintains luminance longer than any other color. Even though it is perceived 
as a light caramel color by persons with color blindness, it holds its 
brightness longer than any other hue, and therefore maintains its contrast 
when paired with a dark color. 
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COLOR COMBINATIONS TO BE AVOIDED 

• Avoid gray with any color, even another value of gray. 

• Avoid red with any color except white or blue. 

• Avoid green with any color except white. 

• Avoid brown with any color except white or blue. 

• Avoid purple with any color except yellow or white. 

• Avoid orange with any color except blue or white. 

• Avoid two values of the same color, such as light blue and dark 
blue. 

• Avoid a neutral color with any other neutral color. 

The importance of proper attention to color selection cannot be overlooked 
when developing tests for individuals or groups that have color vision or 
color perception deficiencies. 
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GUIDELINES FOR AUDIO 
VERSIONS OF TESTS 

This section is written to provide assistance in the development and 
implementation of accessible tests for students with visual impairments 
who require audio versions of a test. Audio formats include cassette tape, 
video, CD, computer-based, or spoken (read aloud) test versions. When 
an audio version of a test is administered, the audio version should be 
accompanied by a print, large print, or braille version of the test, or a 
large print or tactile graphic supplement at the very least. In this multi- 
media approach, a student can access illustrations or other visual 
material that may not be described, or only minimally described, on the 
audio version of the test. 

Some illustrations can be described orally in an accurate manner, while 
other graphic material cannot be described without revealing the answer 
or providing an unfair advantage to the audio user. A complete script for 
audio versions should be written with the assistance of a content expert 
and provided to test administrators. 

Audio versions of a test serve to standardize oral delivery of the test 
content and may reduce the number of school staff needed for proctoring 
or administering tests orally. Consideration of these points will facilitate 
the production and administration of test materials in audio format. (See 
Appendix A for a discussion regarding braille versus auditory access.) 

Production of Audio Tests 

1. Test publishers may only have the capability of providing one version 
of a test in audio format. The version selected should be parallel in 
content and difficulty to other versions. 

2. An experienced test editor should be involved in editing for an audio 
presentation of a test. The audio edition will need to be coordinated 
with other media in which the test will be provided. 

3. Audio versions should be developed using the resource The Art and 
Science of Audiotape Book Production published by the National 
Library Service for the Blind and Physically Handicapped, Library of 
Congress. Requirements for narrator, monitor, and proofreader are 
provided in this document. 

4. The National Braille Association in Tape Recording Manual , Third 
Edition (1979) provides instructions for reading mathematics 
instructional materials. This source recommends that graphic 
materials be described, if possible, and accompanied by print or 
tactile versions of the graphics. Such modifications need to be 
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approved by the test publisher. Moreover, the audio descriptions, 
print, and tactile versions of the material need to be coordinated. If 
different departments within the same company or different vendors 
are responsible for developing and producing the accessible media, 
one source should be responsible for ensuring that the media are 
coordinated to the extent specified. 

5. Narration of print materials must follow National Library Service 
(NLS) specifications of minimum acceptable requirements 
(Specifications #300 and #304). 

6. Test publishers must ensure that narrators follow confidentiality and 
security assurance standards of test materials. Security measures 
taken when working with audio formats should mirror those required 
for handling print or braille materials (Kentucky Core Content Test 
Administration Manual Supplement, 2000). 

7. Test publishers must give attention to packaging and labeling of the 
audio test. Audio tests may be packaged for each individual student, 
with the appropriate print and/or braille supplements needed by the 
student and the test administrator. 

8. Ascertaining whether the audio format will be administered on an 
individual basis or in a small group setting is important, as the 
information on the audio format and the information to be provided 
by the test administrator may vary depending on the setting. 

9. The audio test should instruct the student to stop at certain 
points. Audio procedures must ensure that test takers work only on 
allowable sections of the test. For example, selected subtests may be 
recorded on separate cassette tapes or CDs and then collected as 
required. 

10. Directions for navigating through the audio version should be 
provided in print for the test administrator. 

11. Test publishers must select an experienced narrator with appropriate 
voice, speech, accuracy, and pronunciation skills. (Pronunciation 
resources are available from NLS.) 

12. Narration must be evaluated and proofread to ensure that test 
content is conveyed accurately and that questions are presented 
without unintended emphasis on correct answers. 

13. A person with identical test materials should monitor the narrator to 
ensure accuracy during audio production. A third person should be 
used for the proofreading of audio materials. 
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Administration of Test Items in Audio Formats 

1. Students using audio versions of a test should have had an adequate 
amount of experience using the specific audio medium and audio 
equipment independently before the testing situation. 

2. Test administration materials should indicate the equipment 
required by the student for using audio versions of a test. For a 
cassette tape version of a test, a standard two-track tape player/ 
recorder and headphones will be needed. An audio test on CD will 
require use of a common CD player or a digital talking book player, 
depending on the audio file type of the CD. Regardless of the player 
used, a backup player capable of playing the same audio medium 
should be available. Access to electrical power or sufficient batteries 
for player/recorder use should be indicated. Test administrators 
should be instructed to inspect the equipment functions before 
testing begins. 

3. Test administrators will need to monitor student "movement" 
through audio versions to ensure that the student maintains the 
appropriate place in the test and to ensure that the audio version is 
playing properly. When using a two-sided cassette tape, students 
may need to be reminded to play the other side of a tape. Prior to 
administering the test, and in the absence of students, test proctors 
should spot check audio formats to ensure proper operation of the 
audio medium and equipment. 

4. Students using an audio version of a test must be seated in a quiet 
area and away from other students so that other students are not 
disturbed by the audio medium or equipment operation. Students 
can choose to use headphones. 

5. Provisions must be made in the test administration manual for the 
malfunctioning of audio equipment. Students may have to be tested 
at a later time if malfunctioning occurs. Students must not be denied 
access to the administration of a test because of equipment 
malfunctioning or failure. 
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GUIDELINES FOR ORAL READING OR 
SIGNING OF A TEST 

Students who are visually impaired or deafblind may need the accom¬ 
modation of a reader or sign language interpreter. Occasionally, an audio 
version of a test is not produced, and a test publisher, developer, or 
assessment personnel will allow the reading or signing (use of sign 
language) of a test or portions of a test for students whose Individual¬ 
ized Education Program (IEP) specifies this accommodation. Before using 
oral reading or sign language as an accommodation, careful attention 
must be given to the constructs being measured. For example, if a 
section of the test is designed to assess reading as a decoding skill, then 
the reading or signing of the test to a student would invalidate the results 
for the intended purpose. In these instances, consider an alternate test 
or redefine the construct for the individual student. Always check with 
the test publisher or test developer to determine the construct intent 
and accommodation use for particular sections of a test. 

State policy dictates if passages and stimuli can be read aloud and/or 
signed for large scale statewide assessments. Check with the District Test 
Coordinator or with the State Department of Education assessment office 
for the policy in your state. 

For the oral reading or signing accommodation to be allowed on statewide 
assessments, a student must have had exposure to and have used this 
accommodation during daily instruction and on classroom tests. This is 
especially true when mathematical symbols and technical or content- 
related language is being read and accessed. It is recommended that a 
student have access to print or braille graphic material even if the reading 
or signing accommodation is used. 

The Educational Testing Service recently posted on their web site ETS 
Guidelines for a Test Reader (July, 2000), which have been made available 
in Appendix G of this document through special permission from ETS. This 
document is helpful in outlining the characteristics of a good reader, 
providing general information for readers, indicating special considera¬ 
tions for multiple-choice tests, addressing mathematics reading, and 
providing test center procedures for using a reader. In addition, 
consideration of the following points will ensure appropriate provision of 
oral reading or signing of a test or portion of a test: 

1. Test security and confidentiality standards must be upheld. 
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2. The test purpose must be specified to ensure that reading or signing 
a test or portions of a test do not invalidate results or preclude how 
the results will be reported. 

3. An experienced test editor and professionals involved in working with 
students who require readers or interpreters need to be included in 
the team of persons that adapt tests which are to be read or signed. 

4. A prepared script must be provided for test administrators to ensure a 
consistent, standardized presentation of the test items. 

5. A reader or sign language interpreter must have skills in 
presenting various types of test materials. For example, someone 
familiar with mathematical symbols is needed in order to correctly 
read and convey higher level math formulas and equations. 

6. A standard video presentation of the test in sign language is 
recommended to ensure quality, consistency, pacing, and accuracy. 

7. The person selected to read a test to a student should have the 
characteristics of good voice quality and appropriate speed and tone. 

8. The person signing a test must be a trained interpreter and be able 
to translate in the same method of sign language typically used by 
the student. It is not recommended that the student's teacher be 
the interpreter for the testing situation unless a second person is 
present to monitor for quality and fairness during administration of 
the test. 

9. Voice inflection (regional dialect and pronunciation) familiarity is 
recommended. 

10. The narrator or interpreter must avoid voice inflection that stresses 
or otherwise indicates the correct answer. 

11. The interpreter must avoid facial expressions and body language that 
may cue the correct response. 

12. Students tested through oral reading of the exam must be tested 
individually to prevent the testing situation from becoming a group 
effort. Moreover, testing individually helps ensure that each student 
receives the specific oral reading structure required by his or her 
individual needs. 

13. Directions can be read or signed to groups of students. 

14. The interpreter or reader must be allowed to review test 
administration materials and items on the test to ensure that they 
have knowledge of the vocabulary/signs required for that 
assessment. This is important so that the reader/interpreter does 
not accidentally cue the correct response. The reader/interpreter 
should have access to pronunciation dictionaries, sign language 
dictionaries and technical skills manuals to use as references. It is 
important that the reader/interpreter sign a confidentiality agree¬ 
ment before reviewing the materials to ensure test security. 
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15. Oral readers and interpreters will need to pause at appropriate 
intervals to provide the student an opportunity to answer test items 
or access graphic material provided in print or tactile formats. 

16. Graphic materials may be described as detailed in the prescribed 
script, but must also be made available in print or tactile formats. 

17. Oral readers or interpreters must avoid providing an answer to a 
student's question concerning clarification of testing content. Doing 
so would provide an unfair advantage. Developing some standard 
responses to students' questions prior to the testing situation is 
helpful. For example, you can encourage the student to listen to or 
watch the signing of the question again. 

18. Readers or interpreters may need to provide multiple readings or 
signings of passages, parts of passages, or items. Unless instructed 
otherwise in the Test Administration Manual, professional judgment 
and any guidance provided in the IEP should be used to determine 
the number of readings necessary. 

19. If the oral reader or interpreter is also completing an answer sheet 
for a student, the transfer of answers must be performed carefully to 
ensure that the student's answers are recorded as intended. See 
section on Guidelines for Braille and Large Print Test Response 
Transcription. 

20. Two readers or interpreters should be used for presenting a test or 
portions of a test to a student. Using two readers or interpreters 
helps ensure accuracy of test presentation and provides the 
opportunity for readers or interpreters to rest after 15-20 minutes 
of presenting test material. 
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ACCOMMODATIONS IN TESTING STUDENTS 
WITH VISUAL IMPAIRMENTS 

The use of accommodations during testing is intended to level the 
playing field for any student with a disability. There are, by nature of 
the disability, certain accommodations that are needed by students 
with visual impairments. Not all of them discussed in this section are 
intended for use by all students with visual impairments. Likewise, 
some needed by students with visual impairments may not be 
presented here. 

Accommodations and various technologies exist to provide learners 
with visual impairments access to academic instruction and tests. The 
term "technology" comes under the definition of assistive technology 
as described in federal law and is considered an accommodation to the 
testing of students with visual impairments. 

The need for one or more accommodations is the decision of the 
Individualized Education Program (IEP) team and must be recorded on 
a particular student's IEP. Accommodations used during testing should 
generally match those used by the student for classroom instruction, 
assuming they are familiar and effective for the student. Their use is 
determined by evaluating factors unique to each student and must be 
implemented as outlined on the IEP. Evaluation of their effectiveness 
for an individual student is highly recommended. Further, students 
must be trained to use accommodations. For example, providing a 
test orally by a qualified person or on computer might actually penalize 
a student who has not been trained to listen to orally presented 
material or trained to use a computer for assessment. 

Accommodations should be periodically evaluated to ensure that they 
are still effective for the student. Some may need to be eliminated or 
revised when and if the student arrives at a point where he or she 
either does not need the accommodation, it is ineffective, or it is not 
the most effective option available. If an accommodation is needed by 
a student and is not on the list of those approved for state use, the 
local test administrator should contact the state assessment office to 
request a review of its use. 

The next segment presents general as well as specific accommodations 
for test takers with visual impairments who use braille, large print, 
and/or audio formats. See Appendix F for additional information on 
this topic. 
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Types of Accommodations for Students 
with Visual Impairments 

Presentation Accommodations 

1. Braille, large print, and audio are accommodations that some 
students with visual impairments will use interchangeably. A student 
may, for example, read a passage in braille and prefer to access a 
table or chart in an enlarged version of the test. Therefore, students 
should be allowed to use a large print (or regular print with 
magnification) and a braille version of the test, if requested. 

2. Some students who are visually impaired may need to have read to 
them the test directions or some of the test items, as long as those 
items read do not assess reading as a decoding skill. See section on 
"Some Guidelines for Oral Reading and Signing of a Test. " 

3. Computer-administered testing is an accommodation that has 
received some attention through research, though studies 
concerning its benefit are inconclusive (Tindal & Fuchs, 1999). 
Generally, however, when a student uses a computer for daily 
classroom activities, then this accommodation may prove useful 
during testing if the concepts being tested are not undermined. There 
are several programs and peripheral materials that can be used to 
adapt the computer for use by persons with visual impairments. 
Screen readers, text to speech technology, and keyboard access 
through braille or switches are available. Depending on the construct 
being tested, test administrators must verify that the student is 
inhibited from accessing software or hardware that may provide an 
unfair advantage. For example, if a student's basic math skills are 
being assessed and the intent is not to use a calculator, then the 
keyboard functions or software used for computations must be 
blocked. 

4. When testing allows the use of non-scientific or scientific calculators, 
students with visual impairments should be permitted to use an 
equivalent device that has been adapted for use by a visually 
impaired user. Should a state provide non-scientific or scientific 
calculators for the sighted population taking the test, then non- 
scientific or scientific, talking calculators should be provided to 
students with visual impairments who are taking the test. 

5. An abacus is often useful for students when mathematics problems 
are to be worked without a calculator. The abacus functions as paper 
and pencil for some students with visual impairments who have 
received instruction and practice on the use of the abacus. See 
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Appendix D for the position paper "Use of an Abacus in Test-Taking 
Situations." 

6. Students may want to use manipulative devices, such as a ruler or 
template, to maintain placement on a line of braille or print. Other 
tools available for use by visually impaired students include braille or 
large print rulers and protractors, raised line or bold line graph 
paper, or raised line or bold line writing paper, to name a few. 

Contact the American Printing House for the Blind (APH) toll free 
at 1-800-223-1839 to request a catalog of available accessible 
materials, or visit APH on the Internet at www.aph.org. 

Response Accommodations 

1. Students with visual impairments may need to present answers 
orally to a test administrator who completes the answer sheet. See 
section on Guidelines for Braille and Large Print Test Response 
Transcription. 

2. Students with visual impairments may need to write answers in the 
test booklet or on separate paper using a braillewriter or slate and 
stylus. 

The student's answers must then be transcribed and transferred to 
the answer sheet. See section on Guidelines for Braille and Large Print 
Test Response Transcription. 

3. Students may need to write answers using a word processing 
program, to be transferred to the answer sheet. Depending on the 
construct being tested, test administrators must verify that students 
are inhibited from accessing software or hardware that may provide 
an unfair advantage. For example, if a student is responding to a 
writing prompt and the writing will be judged based on correct 
spelling and grammar, then the spell check function and grammar 
functions must be disabled. 

4. If a student must draw or somehow demonstrate a response, then 
accessible tools and materials that are typically used by the student 
for instructional purposes must be made available in the testing 
environment as long as no unfair advantage is provided. For this 
type of open response item, it is very important that scoring criteria 
be well defined and allow for variation in response methods. 

Timing Accommodations 

1. The use of extended time for test completion is a testing 

accommodation that has received considerable attention since state 
testing and accountability systems have been implemented. Research 
investigating the use of extended time has yielded little conclusive 
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information about its benefit (Tindal & Fuchs, 1999). However, 
students with visual impairments will usually require extended time 
during testing because using braille, print, and audio formats require 
more time than does reading print with acceptable visual acuity. A 
study by Wetzel and Knowlton (2000) suggests that experienced adult 
braille readers may need no more than 50% more time than the 
stated duration, with additional time allowed for the manipulation of 
an audio device or the marking of an answer sheet. In contrast, an 
earlier researcher found that braille readers with far less braille 
reading experience than the subjects mentioned in the Wetzel and 
Knowlton study may need between 2 and 3 times as much time as 
their sighted peers to read the same material (Nolan, 1966, p.l). 
Traditionally, extended time for testing readers who are visually 
impaired has been IV 2 times, and for braille readers time allotted has 
been 2 times the amount allowed for regular print readers 
(Lowenfeld, Abel, & Hatlen, 1969, pp. 91-92). Regardless of the time 
allowed, the student should be carefully monitored to ensure that time 
is being used appropriately. If students need an inordinate amount of 
time, educators may need to investigate the efficiency of the chosen 
reading mode or initiate remediation to improve speed. Generally, 
timing accommodations should be individualized according to the test 
taker's reading rate and testing situation (Wetzel & Knowlton, 2000). 
See Appendix E on the "Use of Extended Time." 

2. Reading braille, print, or listening to material presented orally, 
especially when accompanied by graphic material, can be a fatiguing 
and often frustrating experience in a high stakes testing environment 
for students with visual impairments. Therefore, students may need 
several brief sessions in which to take the test. Additional break 
options should also be considered. 

3. Students may need to be tested over a longer time period, a week 
rather than two days, for example. However, any alteration of the 
timetable will necessitate close supervision to ensure test security. 

4. Students may need to be tested at different times of the day 
depending on their optimal functioning time. 

Setting Accommodations 

1. Some students with visual impairments may need to be administered 
a test or select subtests individually, or in small groups as 
recommended on their IEP, to ensure that the test accommodations 
needed by the students are implemented without interfering with 
the concentration and test taking results of other students. 

2. If a student is recording answers by using technology that is noisy or 
is recording answers orally, then he or she must take the test 
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individually and under the supervision of a test administrator in order 
to avoid distracting or influencing the responses of other students. 


Specific Accommodations in Testing Readers Who 
Require Enlarged or Large Print 

Enlarged print is that which is 14 point, 16 point, or regular print that 
has been enlarged using magnification devices. Large print is 18-point 
type and larger. Enlarged print and large print are accommodations. 

Some students may choose to use a regular print test and enlarge it 
manually with a magnification device with which they are familiar. 
Magnification devices include eyeglass-mounted magnifiers, free standing 
or handheld magnifiers, and electronic equipment such as the closed 
circuit television (CCTV) or a computer that has text enlargement 
software installed. These devices do not provide a student with an unfair 
advantage. Rather, they are devices that the student requires to access 
print, and they should be allowed as standard accommodations. 

Proper lighting and freedom from glare, while sometimes overlooked, are 
critical for many readers with visual impairments. Lighting that has been 
adjusted to suit the student's particular visual needs and minimize glare 
will help promote sustained reading efficiency. 

Specific Accommodations for Audio and Oral Test 
Administration 

Students using an audio version of a test or having the test orally 
administered as an accommodation should also be allowed to have print 
(large print or regular print with a magnification device) and braille 
versions of the test, if requested. A student may wish to listen to a 
passage by way of audio, but access a table or chart in a large print or 
braille version of the test. Listening to an oral description of a geometric 
figure can be difficult or impossible to follow unless an enlarged graphic 
or a tactile graphic accompanies the oral description. 
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GUIDELINES FOR BRAILLE AND 
LARGE PRINT TEST RESPONSE 
TRANSCRIPTION 

Some students with visual impairments will use the accommodation of 
oral response, written response (on the test booklet or on paper other 
than the test answer sheet provided by the test publisher), or taped 
response. Each of these accommodations requires that a person 
transcribe the answers onto the answer sheet or booklet that will be 
scored. These guidelines are provided to ensure that transcription is 
performed appropriately. 

1. Confidentiality of the test materials and the student's individual 
responses is critical. Transcribers must treat the testing materials 
and the student responses in a secure and confidential manner to 
ensure test and student identification security. 

2. Response transcribers must know braille if transcribing braille 
responses. 

3. It is best if the response transcriber is a "neutral" person, not 
someone with a vested interest in the student's scores. 

4. Response transcribers must provide the exact answers that the 
student has written using the same punctuation, spelling, and 
grammar structure. They cannot guess what the student might 
have meant if answers are incomplete. 

5. It is recommended that the response transcriber have a second 
person proofread the responses to ensure accuracy and fairness to 
the student. When transcribing graphics that a student has 
produced, two transcribers should work together in transferring 
student answers to the answer sheet or booklet. 

6. For a period of time, student responses must be maintained in a 
secure file with test name, copyright year, form and level 
administered so that the student's actual responses can be 
reviewed if questions arise. 
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REPORTING TEST RESULTS OF STUDENTS 
WITH VISUAL IMPAIRMENTS 

Following the requirements of federal law, the scores of students who 
take assessments in accessible format must be reported for account¬ 
ability purposes. When reporting the results of students with visual 
impairments, care must be taken to protect the student's privacy while 
appropriately representing the test score in consideration of the 
accommodation(s) used. Students must not be penalized for use of 
approved accommodations that do not change the test construct and do 
not provide an unfair advantage to the test taker. Reporting of scores 
should be a consideration during the test development phase so that all 
parties understand the purpose of the testing and how the results will be 
reported and used. 

Reporting Test Results for Braille Editions 

For most assessments, braille test versions should be regarded as 
appropriate accommodations for students who use braille daily. Any 
rescaling of braille test versions that is performed because of item 
omission should be reported. The scores of those students taking a test in 
braille should be considered valid as long as the test has been prepared 
using the guidelines presented in this document. Students who read 
braille daily need to use braille to respond to test items. This dual use 
provides an instructional/ assessment validity match. Extensive efforts 
to "prove" a braille test invalid because of a difference in format are 
neither recommended nor useful. If the purpose of a test is to determine 
educational skill progress, the validity can be addressed by confirming 
that the media used for instruction matches that which is used for 
assessment. 

Reporting Test Results for Large Print Editions 

Large print versions of tests also qualify as appropriate accommodations 
for use during the assessment of students who use large print daily. 

Unless the assessment has been reformatted, the large print version is a 
camera-enlarged version of the original version. If the test is altered 
through removal of shading, or other clutter from graphics, the use of 
the large print format should be considered an appropriate and valid 
accommodation. Generally, if reformatting is performed in a manner 
preserving the original test content, the reformatted version should be 
considered valid. Producers of large print must work with test publishers 
to verify that the test material has not been altered in content or 
purpose in order to maintain test validity. 
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Reporting Test Results for Audio and 
Orally Administered Tests 

Regarding most assessments, the use of audio and orally administered 
tests should be considered appropriate accommodations for students who 
use audio and oral formats on a routine basis to access materials. For 
tests that assess reading as a decoding skill (visually or tactually), audio 
and orally administered versions may change the skill being tested, and 
this should be noted in any report of scoring. 
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ALTERNATE ASSESSMENTS 

The guidelines presented in this section address some specific issues 
related to accessibility of alternate assessment for students who are 
blind or visually impaired. The guidelines addressing general state 
testing that are presented throughout this book are appropriate 
considerations for providing alternate assessment materials for some 
students in this population; however, the specific needs of this group 
must be discussed because so many are non-readers. Best practices in 
this arena are still being formulated. 

General Issues 

Students who meet the criteria for alternate assessment, by definition 
of the federal law, are those students who have significant cognitive 
disabilities (often referred to as the 1% population assessment). 

As allowed by federal law, some states have chosen to provide a 
second alternate assessment for those students who are not expected 
to meet the state standards as demonstrated on the general state 
assessment within the same time frame as students taking the general 
state assessment. In addition, these students are to be working 
toward the state standards using modified achievement standards as 
identified by each state. In some states, this alternate assessment 
(generally referred to as the 2% population assessment) mirrors the 
general state assessment with the exceptions of having fewer answer 
choices and in some cases using simpler language in the test items. 

Since alternate assessments are very similar to the general state 
assessment in most cases, the same requirements for accessibility are 
applicable for the alternate assessment as are outlined for the general 
assessment. 

The needs of students who are blind or visually impaired and have 
additional disabilities that may qualify them for these alternate 
assessments, must be considered in the planning and developing of 
alternate assessment formats and items. Providing accessibility for this 
population of students requires that test publishers and state 
personnel have access to professionals who are familiar with braille, 
large print, and regular print and know the learning styles of these 
students. 

Because many students who take alternate assessment have limited 
reading ability, it is expected that students who are blind who qualify 
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for alternate assessment will have very limited braille reading 
capabilities as well. Likewise, students with low vision who qualify for 
alternate assessment may have limited ability to read print or large 
print. 

While providing a general assessment in braille for accessibility 
purposes has its challenges, the provision of an alternate assessment 
in tactile format can be even more challenging. Alternate assessments 
typically have formats that either require the student to answer 
questions by looking at a visual stimulus or demonstrating skills from a 
checklist of desired tasks. Alternate assessments often include 
performance tasks, such as picture identification or demonstration of 
skills using manipulatives. Generally, demonstration of specific skills 
on alternate assessments can be easily accommodated to allow the 
student who is blind or visually impaired to perform tasks in the usual 
way they perform tasks in the classroom. Validity and reliability can 
usually be maintained when accommodations have been well- 
documented on a student's IEP and assessment report. 

Considerations in Alternate 
Assessment Design 

• Because reading is an issue for the population of students taking 
alternate assessment, test items often require picture identification. 
Test administrators must be able to describe the pictures for 
students who are blind. All test items need to be reviewed to assure 
that the picture can either be described (accessible) without giving 
the answer away or that the picture is not needed (inaccessible) 
and has been omitted. Keeping the task appropriate to the student 
who takes the test is crucial: For example, it is appropriate to ask a 
student about the function of an object (i.e. Which of these can you 
eat— a book, a rock or a banana?). An item would not be accessible 
if it asked: "Which of these pictures shows a banana?" In this later 
example, naming the pictures (book, rock, banana) would give the 
answer away. 

• It is recommended that the test publisher provide picture descrip¬ 
tions for the test administrator. Picture descriptions should be 
developed in conjunction with content experts and state 
assessment personnel, keeping in mind the construct (skill) being 
assessed and the cognitive level of the students taking the test. 

• If pictures cannot be described without compromising the test item, 
it is preferable to present manipulatives (objects) to students in lieu 
of pictures. If this is allowed by the state, such objects should be 
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real objects (i.e. actual familiar classroom objects and shapes such 
as pencils; paper; books; notebooks; toys; food; geometric shapes; 
and counters) and not replicas. 

• Replicas or miniatures of animals or of other large objects are not 
appropriate for use because they cannot be distinguished by the 
student. These should be used with caution and only if the student 
is familiar with the models or miniatures. 

• Real money should be used rather than a tactile representation. 

• Tactile representation of simple graphs (charts) is appropriate. If 
the test administrator is allowed to read the chart or graph, a script 
for reading it is preferred and should be included in the test 
administration manual. 

• Tactile representation of shapes (circles, squares, stars, rectangles, 
etc.) is appropriate and should be used in place of letters, animals, 
and people to present counting or other mathematics items. 

• If the skill being assessed is not reading, then test administrators 
should be allowed to read aloud all words and passages used in the 
test items. Care must be taken not to give vocal clues by 
emphasizing certain words. 

• If the construct being assessed is decoding of words (i.e. reading) 
and simple words and/or passages are provided, then these words 
and passages must be provided in braille. It is important for state 
assessment personnel to determine if contracted or uncontracted 
braille, or both will be provided. This can best be done by surveying 
a sample of teachers who work with this population to determine 
which format is preferred or by requesting school districts to specify 
the number of alternate assessments that are needed in contracted 
braille and the number needed in uncontracted braille. Making this 
decision part of the ordering process ensures that the appropriate 
braille test format is provided for each individual student. 

• Large print is defined by research as optimal at 18 points. A sans 
serif font should be used for best readability. 

• It is important that any pictures or graphics provided in print or 
large print are clear, uncluttered, black line drawings with no grey 
scale. 
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APPENDIX A 

BRAILLE VERSUS AUDITORY ACCESS: A 
DISCUSSION 

Federal law requires that consideration be given to accommodations in 
testing students with disabilities. With this focus comes the responsibility 
of the educator to identify needed and useful accommodations for 
students with disabilities. For students with visual impairments, 
accommodations that provide access to print can vary considerably. The 
range of accommodations includes braille, tactile graphics, large print, 
regular print with magnification, auditory media, or any combination of 
these accessible media. This discussion suggests methods for identifying 
the most appropriate accessible media, identifies uses of braille and audio 
materials, and provides recommendations for consideration in choosing 
testing media. 

Since the early 1990s authors have identified methods of evaluating the 
"mode of reading" or method of print access for students with visual 
impairments (Koenig & Holbrook, 1993; Wormsley & D'Andrea, 1997). 
Federal law indirectly requires that print access be evaluated by defining 
the consideration of braille as a mode of reading for students with visual 
impairments as part of the Individualized Education Program (IEP) 
process. A major part of the early and ongoing assessment of a visually 
impaired student's unique needs is the use of various media to access 
printed materials. Identification and use of appropriate media includes: 

• Determination of the student's primary and secondary sensory 
channels for learning through observation of the student's use of 
vision, use of touch, and use of hearing in familiar and unfamiliar 
settings, at structured times and unstructured times, and in outdoor 
settings as well as indoor settings (Koenig & Holbrook, 1993) 

• Attention to the student's current print access needs, instruction and 
remediation in accessible media or alternate media, and recognition 
of future needs in print access for the student 

• Provision of initial sensory channel identification and ongoing sensory 
channel use to determine changes in use and need for instruction in 
additional media access skills 

• Instruction in a variety of accessible media that could be used by the 
student 

• The opportunity to learn skills that enable the student to choose the 
appropriate medium for various tasks 
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Specifically, the appropriate uses for braille are determined by each 
individual who uses braille. Most blind individuals access printed materials 
by using a combination of media. One issue in using braille has typically 
been the lack of braille materials. Currently, there are improved methods 
of providing braille materials through the expansion of technology. 
Computer software and hardware that translate print to braille, provide 
braille displays, and emboss braille through a translation program are 
used to provide most braille text in a timely fashion. It should be noted 
that print with highly graphic and technical content does not translate to 
braille easily and with the type of accuracy expected for testing 
materials. 

Congress has recently passed legislation that will ensure accessibility of 
instructional materials in braille for students with visual impairments. 
While assessment materials are not included in this legislation, it seems 
that making instructional materials readily available and accessible will 
drive the need for a similar pattern in the testing arena. 

Persons with visual impairments routinely use auditory means to access 
large volumes of literary or recreational reading material, such as novels 
or magazines. The expansion of technology and the ability to translate 
printed text into speech has enabled persons with visual impairments to 
access information via computer software and/or hardware. Additionally, 
many persons with a visual impairment make use of a screen reader for 
print access, a skill that requires some training. 

The availability of a wide range of ways to access print is important for 
persons with visual impairments. This range of availability should exist 
for students but should not be confused with, or used as a replacement 
for, the skill of learning to read (decode language). If society values the 
reading of materials as a decoding skill, then access to printed material 
for students who are visually impaired must include the learning of 
reading through tactual or visual processes. For some individuals the 
reading process is too tedious to be efficient. These individuals may 
choose to use primarily auditory materials as adults, but as students 
they should be given the opportunity to learn reading as a decoding skill. 

The skills involved in reading braille, reading print, and listening to audio 
materials are unique to each medium. Therefore, during the development 
of test items, test publishers must be clear about which constructs are to 
be assessed by a particular item. If reading as a decoding skill is to be 
assessed, then a fair assessment can only result if the student is 
provided with material that can be visually or tactually read. If 
comprehension is the construct being assessed, then the test developer 
must determine whether reading comprehension or listening 
comprehension is the skill to be assessed. Comprehension would need to 
be defined to ensure that students are using appropriate accommodations 
when taking a particular test. 
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The following recommendations should be reviewed when considering the 

use of braille or audio materials for students with visual impairments: 

1. Braille and tactile graphics interpretation should be taught as media 
access skills so that students may learn reading as a decoding skill 
and have the option of using braille and tactile materials. 

2. Auditory listening skills should be taught as a media access skill so 
that students can learn listening comprehension skills and have the 
option of using audio materials. 

3. Test publishers must be certain about the construct being assessed 
on all test items. This enables educators and test administrators to 
make valid judgments about appropriate accommodations for 
students with visual impairments during test administration and helps 
to ensure correct interpretations of test results. 
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APPENDIX B 

TEMPLATE FOR TEST ADMINISTRATION 
NOTES FOR BRAILLE TESTS 

Name of Test: 


Edition of Test: 


Section: 


Preliminary Pages Transcriber's Notes: 


Special Symbols Page: 


General Test Direction Notes: 


Print Page 
Number(s) 

Braille Page 
Number(s) 

Accompanying 

Test 

Administration 
Manual Page 
Number(s) 

Item 

Number(s) 

Notes 
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Explanation of Fields on 
Test Administration Notes for Braille Tests 


Name of Test: Provide the full and exact name of the test. 

Edition of Test: Provide the copyright or other edition listing to further 
identify the test. 

Section: Provide the section name and other identifying information. 

Preliminary Pages Transcriber's Notes: Provide in print the exact 
wording of transcriber's notes that refer to preliminary pages in the 
braille version of the test. Indicate the page number of the transcriber's 
notes. 

Special Symbols Page: Provide in print the exact wording of the special 
symbols page that may be present within the braille version of the test. 
Indicate the page number of the special symbols page. 

General Test Direction Notes: Provide information about the methods 
a student may use when responding to test items that differ from print 
test versions and which require special equipment or attention. 

Print Page Number(s): Provide the location of test material within the 
regular print version of the test. 

Braille Page Number(s): Provide the location of test material within 
the braille version of the test. 

Accompanying Test Administration Manual Page Number(s): 

Provide the page number(s) in the test administration manual that 
correspond with each regular print test page. 

Item Number(s): Provide the test item number(s) that appear on that 
print page. 

Notes: Provide comments that indicate transcriber's notes specific to 
particular pages, changes made to the braille version of the test, and 
changes made to directions, as listed in the test administration manual or 
on the test. 
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APPENDIX C 

TEMPLATE FOR TEST ADMINISTRATION 
NOTES FOR LARGE PRINT TESTS 

Name of Test: 


Edition of Test: 


Section: 


General Test Direction Notes: 


Print 

Page 

number(s) 

Large Print 

Page Number(s) 

Accompanying 

Test 

Administration 

Manual 

Page Number(s) 

Item 

Number(s) 

NOTES 
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Explanation of Fields on 

Test Administration Notes for Large Print Tests 


Name of Test: Provide the full and exact name of the test. 

Edition of Test: Provide the copyright or other edition listing to further 
identify the test. 

Section: Provide the section name and other identifying information. 

General Test Direction Notes: Provide information about the methods 
a student may use when responding to test items that differ from print 
test versions and which require special equipment or attention. 

Print Page Number(s): Provide the location of test material within the 
regular print version of the test. 

Large Print Page Number(s): Provide the location of test materials 
within the large print version of the test. 

Accompanying Test Administration Manual Page Number(s): 

Provide the page number(s) in the test administration manual that 
correspond with each regular print test page. 

Item Number(s): Provide the test item number(s) that appear on that 
print page. 

Notes: Provide comments that indicate changes made to the large print 
version of the test and changes made to directions as listed in the test 
administration manual or on the test. 
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APPENDIX D 
POSITION PAPER: 

USE OF AN ABACUS 
IN TEST-TAKING SITUATIONS 

By Terrie Terlau and Fred Gissoni 

Definition and Description 

The mathematical abacus is a frame with beads or balls that can be slid on 
wires or in slots for calculating or teaching arithmetic (The American 
Heritage Dictionary of the English Language, 1996). The abacus has been 
used as a calculation device in Europe, Japan, China, and the Middle East 
since the third century A.D. It continues to be used widely in Japan 
( http://www.svuzan.net/enQlish/education/education.htmn . 

The Cranmer abacus was developed as a calculation device for persons who 
are blind or visually impaired and is currently produced by the American 
Printing House for the Blind (APH: Abacuses, 2001). The Cranmer abacus 
frame is made of high impact plastic, measures 6-1/8 x 3-1/4 x 7/16 inches, 
and contains thirteen vertical rods and one horizontal cross bar. Four beads 
can be moved vertically on each of the thirteen rods below the cross bar and 
one bead can be moved vertically along the rods above the cross bar. 

Abacus Functionality 

When calculating with the Cranmer abacus, vertical rods represent units, 
tens, hundreds, etc. Numbers are recorded and manipulated by moving 
beads toward the cross bar on their respective rods. 

The abacus is a passive device. It is not a calculator or a slide rule. The 
abacus does not perform mathematical operations. It does not contain 
information that would enable an abacus user to achieve calculation results 
without a solid knowledge of mathematical concepts and relationships. 
Abacus users produce calculations as a result of their understanding of the 
behavior of numbers, not because of any inherent property of the abacus. 

Both abacus and pencil-and-paper users must learn strategies for performing 
mathematical operations. The primary difference in the activity of abacus 
and pencil-and-paper users is that pencil-and-paper users apply and record 
steps in these operations by writing while abacus users apply and record 
these processes by moving abacus beads. 
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Persons who are blind or visually impaired and who have had appropriate 
abacus instruction can use the abacus to perform addition, subtraction, 
multiplication, division, and square and cube roots. The abacus does not 
permit permanent storage of problem solutions because beads must be 
rearranged to perform subsequent problems. After each calculation using an 
abacus, answers can be recorded in a variety of formats including braille, 
large print, voice recording, word processing, or dictation into an electronic 
device. 


Position Statement 

Whenever a test-taker is allowed to use a pencil and paper for working 

calculations, an abacus should be considered an equivalent substitution. 
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APPENDIX E 
POSITION PAPER: 

USE OF EXTENDED TIME 

Introduction 

In addition to the use of braille and large print, the use of extended time 
is also a commonly used accommodation for students with visual 
impairments. This position paper provides a brief summary of the results 
of research on the use of extended time in testing students, while 
suggesting best practices for implementing this accommodation. 

Research 

For several years, researchers have suggested that students with a visual 
impairment need more time to complete assignments and tests (Harley & 
Lawrence, 1984; Kederis, Nolan, & Morris, 1967; Morris, 1974; Spungin, 
2002; Bradley-Johnson, 1994). 

Moreover, some researchers have reported results indicating that 
students with a visual impairment generally read at a slower rate than 
students without a visual impairment (Packer, 1989; Legge, et. al., 
1985, 1989; Wetzel & Knowlton, 2000). Not only does the reading of 
braille and large print generally require more time than reading regular 
print, but the time needed to explore and interpret pictorial information 
presented as tactile or enlarged graphics can be a tedious and time- 
consuming process. Therefore, extended time seems to be an obvious 
accommodation for this population. Some suggested time extensions 
based on classroom experience or research include 

• 1.5 to 2 times for students with low vision (Gompel, van Bon, & 
Schreuder, 2004), 

• 2.5 times for braille and 1.5 times for large print (Morris, 1974), 

• 1.5 times for all students with a visual impairment (Spungin, 2002), 

• 2 times for braille (Kederis, Nolan & Morris, 1967), 

• More than 2 times for braille and a little less than 2 times for visually 
impaired readers who read print (Packer, 1989), and 

• .5 times for experienced adult braille readers (Wetzel & Knowlton, 
2000 ). 

The most recent synopsis of research on accommodations demonstrates 
the wide range of results among studies seeking to validate the use of 
extended time during testing. Based on the varied results, authors 
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recommend that a well-designed test for standard administration be 
untimed (Tindal & Haladyna, 2002). 

Research conducted by the National Center on Educational Outcomes 
(NCEO) summarizes at least four studies in which the use of extended 
time had a positive effect on student test scores. NCEO provided 
preliminary results of a Universal Design Research project which suggest 
that unlimited time reportedly helps students "think better," a conclusion 
drawn after interviewing students who had completed a universally 
designed test (with no time limits) and a regular test (with time limits) 
(Presentation: Universal Design Research, C. Johnstone & A. Morse, June 
24, 2003 at CCSSO Large Scale Assessment Conference, San Antonio, TX). 

Several authors seem to agree that timed conditions may not allow 
students to reflect their full abilities on achievement tests (Tindal & 
Fuchs, 1999) and that adequate time should be provided for all students. 
Parr, et. al. (1996) argue that extended time examinations taken under 
ideal circumstances can be more equitable and practical than timed 
examinations. In another investigation, Marquart (2000) found that 
extended time failed to significantly improve the test scores of disabled 
students. The author, however, does conclude that extended time likely 
produces a more accurate measure of a student's skill by helping to 
reduce test anxiety and by allowing a greater opportunity to use good 
test taking strategies. 


Conclusions 

Extended time is a commonly used accommodation for students with 
visual impairments. Some literature concerning the subject recommends 
that the accommodation of extended time be of specific duration, e.g., 
2.5 times for braille readers and 1.5 times for large print readers. 
Certainly, a topic in need of additional information is a comparison of 
time used among the following: a braille reader who must explore and 
interpret tactile graphics, a large print reader who must visually examine 
and synthesize enlarged graphics, and a sighted student using regular 
print test materials. Moreover, several current researchers suggest 
placing less emphasis on designating a uniform, "one size fits all" duration 
of extended time as an accommodation for disabled students during 
testing. Rather, these researchers suggest that the accommodation of 
extended time consist of "adequate time." That is, a specific length of 
time, which must be determined by educators through careful 
assessment of the student's physical disability, skills, and needs. In lieu 
of extended time, some test administrators are finding that more 
frequent breaks are effective for braille and large print test takers. Once 
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the need for, and duration of, adequate time and/or breaks has been 
assessed, educators should include that information on the student's IEP, 
ensure use of the accommodation, and monitor its use. 

Position Statement 

To implement extended time or adequate time for students with visual 
impairments, four basic steps should be followed: 

1. Assess the need for extended time and frequent breaks. 

2. Include specific information about extended time and the need 

for breaks on the student's Individualized Education Program (IEP). 

3. Ensure that extended time and frequent break accommodations 
are implemented as specified during testing. 

4. Monitor the student's use of extended time to assure that the 
student uses extended time/break time appropriately and that 
the student is on task. 
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APPENDIX F 
POSITION PAPER: 

ACCOMMODATIONS FORTESTING STUDENTS 
WITH VISUAL IMPAIRMENTS 


By Carol Allman, Ph.D. 

Introduction 

Accommodations and technologies exist for the purpose of providing a 
disabled student with access to academic materials that may otherwise be 
inaccessible. The term "technology" comes under the definition of assistive 
technology as described in federal law and is considered an accommodation. 
Accommodations and assistive technologies needed by students with visual 
impairments should be outlined on the student's Individualized Education 
Program (IEP). These accommodations should be monitored periodically for 
their effectiveness with the individual student and revised or updated as 
appropriate. Any accommodations provided for students during the 
testing window should be ones typically used by that student in the 
classroom and not new or unfamiliar ones. 

This paper provides an overview of accommodations in testing that might be 
effective for students with visual impairments and should be documented on 
their IEP. Five major categories of accommodations that include 
presentation, response, setting, scheduling, and special tools are discussed. 
Not all of the accommodations presented in this paper are intended for use 
by every student with a visual impairment. Likewise, some accommodations 
needed by students with visual impairments may not be discussed. 

Determining Accommodations 

The need for accommodations is the decision of the Individualized Education 
Program (IEP) team and must be recorded on the IEP. Accommodations used 
in testing should match those used by the student for classroom instruction. 
Accommodation use is determined by evaluating factors unique to each 
student and must be implemented as outlined on the IEP. Evaluation of the 
effectiveness of accommodations for individual students is highly 
recommended. Further, students must be trained to use accommodations. 
For example, providing a test orally or on a computer might actually penalize 
a student who has not been trained to listen to material presented orally or 
trained to use a computer for assessment. Accommodations should be 
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continually evaluated to ensure that they are effective for the student. Some 
accommodations should be eliminated if the student arrives at a point where 
he or she either does not need the accommodation or the accommodation is 
ineffective. 


Presentation Accommodations 

Students with visual impairments have several options for accessing test 
materials. According to data collected by the American Printing House for the 
Blind (2003), 9% of the visually impaired student population use braille as 
their primary mode of reading. Approximately 26% use large print materials, 
while only 6% are auditory readers who would require test materials to be 
presented in audio format. Prereaders (27%) may use auditory materials 
until they learn braille or print. Of the nonreaders (32%), some may use 
braille, large print, and audio on a very limited basis. However, there are 
many whose significant cognitive disability would inhibit them from 
successfully using braille, large print, and audio materials. Most of these 
students are involved in educational programs that do not rely heavily on 
traditional reading media and modes of learning and communication. This 
population of students may use augmentative or tactile communication 
systems and might qualify for alternate assessment in the statewide 
assessment program. The remainder of the visually impaired school-aged 
population who are readers access standard print materials with or without 
low vision aids. 

Braille, large print, magnified print, and audio presentation are 
accommodations that allow visually impaired students access to the testing 
environment. Some of these students may use a combination of these media 
to complete a single test. A student may, for example, read a passage in 
braille and prefer to access a table or chart in a large print or magnified 
format. Students using an audio version of a test as an accommodation 
would also be allowed to use print (large print or standard print with a 
magnification device) and/or braille versions of the test, if requested. 

Further, a student may prefer to listen to an orally presented passage but 
access a table or chart in a large print or braille version of the test. If a 
multimedia presentation is used, the various media must be coordinated to 
ensure accuracy and accessibility. It should be noted that computer-assisted 
testing is becoming very popular and requires special attention to be 
accessible for students with visual impairments. 
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Braille and Tactile Graphics 

Braille is a system of raised dots that represent words and letters. It is used 
as a presentation method for those students who typically read braille for 
classroom instruction. Braille may be presented as contracted (using short 
forms for words as outlined in English Braille Code) or in uncontracted format 
(using no short forms, i.e., spelling each word letter by letter). Most students 
will use the standard contracted braille. A few students, such as those who 
are just learning braille in the early grades or who are newly blinded, may 
need uncontracted braille to access a test. 

The production of a braille test is a unique process that often necessitates 
the review and limited editing of test directions and test items so that the 
items are understandable when presented in braille and tactile graphics 
format. Such editing may involve subtle word changes to directions 
(replacing "circle the answer" with "mark the answer"), relocation of stimulus 
information (moving the question above a graph or chart), simplification of a 
graph or chart (removing extraneous information without deleting answers or 
foils), or replacing an item that cannot be reflected in braille with an item of 
equal weight, value, and difficulty (replacing an item that requires strictly 
visual skills, such as visual illusion, with a similar item that assesses the 
same concept and is more accessible to blind students). 

However, an item need not be replaced or omitted simply because it is 
presented in a manner that requires some visual interpretation. For example, 
the concept of understanding a shadow and what causes a shadow is an 
important concept for a blind student to understand. Therefore, this skill can 
be assessed through use of descriptions and tactile graphics. If, however, a 
test or particular subject includes a high percentage of visual items, then 
consideration may be given to substituting some of the "visual" items. 
Students who read using braille are expected to meet the same standards 
that other students meet, even though they are doing so tactually. The 
process of editing a test for braille production should in no way simplify or 
reduce the difficulty of the test material. 

Once test material has been edited for braille transcription, qualified persons 
will transcribe the print into braille by using the recommended edits and 
guidelines for braille transcription and formatting. The transcribed braille test 
must be proofread and produced so that the braille reader receives a high 
quality test in the same timely manner as sighted students receive their test. 
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Large Print Text and Graphics 

Large print is considered such when it is 18-point type and larger. Enlarged 
print is typically that which is 14 point, 16 point, or standard-sized print that 
has been enlarged using magnification devices. Enlarged print and large print 
are accommodations. 

Large print should be produced by using an electronic version of the test to 
reformat the test so that fonts are larger, fewer items are on a page, 
graphics are contained on one page, answer choices are presented with the 
questions, and attention is given to improving the contrast and reducing the 
shading and gray scale that interferes with reading the material presented. 
The process of using a photocopier to enlarge test content should be avoided 
since this method lacks the control needed to ensure that all test material 
(exponential numbers, footnotes, and graphic material) is represented in a 
readable point size, that text is clear and without gray scale interference, 
and that problems dealing with measurement are presented accurately. For 
example, a butterfly measuring two inches in the standard print test must 
remain two inches in the large print version. 

Some students will use magnification devices (discussed in more detail in the 
section of this paper on special tools considerations) with large print or with 
standard print to access test materials. 

Therefore, it is important that the standard print version of a test exhibit 
good contrast and a clear print style to allow effective use of magnification. 

Audio 

Generally, students with visual impairments should be expected to read 
materials by using print or braille. Access to print is a critical literacy skill for 
all individuals. However, where audio presentation is allowed, and for 
reducing the time needed to complete a test, some students who are visually 
impaired may need directions or some test items presented orally to them. 

Audio presentation of print materials is a presentation accommodation 
allowing for all or part of a test to be presented on cassette tape, CD, 
computer and specialized screen reader or text reader software, or read 
aloud to a student. Students should use these accommodations only if they 
use audio media for classroom instruction. The skill of listening to spoken 
material and manipulating a computer, cassette tape player or CD player is 
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different from the skill needed to read and interpret print or braille. 

Therefore, navigating through a cassette tape, computer with screen reader, 
or audio CD in a testing environment requires practice. Further, the test 
purpose must be specified to ensure that oral presentation of a test or 
portions of a test do not invalidate results or preclude the reporting of test 
results. For example, if the reading skill of decoding print (or braille) is being 
assessed, audio presentation of the text could invalidate the purpose of the 
test. 

The transfer of test material onto audio tape requires a process similar to the 
construction of test materials in braille. Print text must be edited for audio 
presentation, produced in audio format by experienced audio engineers, and 
then proofed for accuracy. Additionally, any graphic material must be 
described and provided as a supplement in braille, large print, or standard 
print. Accurately describing graphic material requires attention to the critical 
components of the graphic and careful consideration of which details can be 
included in or omitted from the description without providing the answer or 
excluding the foils imbedded in the question. 

If a test or part of a test is to be read to a student, there are recommended 
practices for ensuring that this accommodation is provided correctly: 

• A reader must be skilled in presenting various types of test materials. 

For example, a reader familiar with mathematical symbols is required for 
the correct delivery of higher level math formulas and equations. 

• The person selected to read a test to a student should exhibit good voice 
quality, appropriate regional dialect, pronunciation, speed, and tone. 

• The reader must avoid voice inflection that stresses or otherwise 
indicates the correct answer to test items. 

• Prior to the testing situation, difficult words within the test material must 
be reviewed by the person assigned to read the test. Pronunciation 
dictionaries should be used as references. 

• Readers must pause at appropriate intervals so that the student has an 
opportunity to answer test items or access graphic material provided in 
print or tactile formats. 

• Readers must avoid answering a student's question concerning 
clarification of testing content. Doing so would provide an unfair 
advantage. Developing some standard responses to students' questions 
prior to the testing situation is helpful. For example, instead of 
answering a student's question about test content, the reader can 
encourage the student to listen to the question again. 

• Readers may find it necessary to provide multiple readings of passages, 
parts of passages, or items. While addressing the needs and requests of 
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the test taker, the reader should also use professional judgment to 
determine the number of readings necessary. 

. Two readers should be used for presenting a test or portions of a test to 
a student. Using more than one reader helps ensure accuracy of test 
presentation and provides the opportunity for readers to rest during the 
presentation of test material. 

• Students tested through oral reading of the exam must be tested 
individually to prevent the distraction of other students. Moreover, the 
testing of students individually helps ensure that each student receives 
the specific oral reading structure required by his or her specific needs. 

Computer-assisted Testing 

Computer-assisted testing is an accommodation that has received some 
attention through research, though studies concerning its benefit are 
inconclusive (Tindal & Fuchs, 1999). Generally, however, when a student 
uses a computer for daily classroom activities, then this accommodation may 
prove useful during testing if the concepts being tested are not undermined. 

There are several programs and peripheral materials that can be used to 
adapt the computer for use by persons with visual impairments. Screen 
readers, text to speech technology, and accessible keyboard access through 
braille or switches are all available. Depending on the construct being tested, 
test administrators must verify that the student is inhibited from accessing 
software or hardware that may provide an unfair advantage. For example, if 
a student's basic math skills are being assessed and the intent is not to use a 
calculator, then keyboard functions or software used for computations must 
be blocked. For more information on this topic, refer to Test Access: 
Guidelines for Computer-Administered Testing. American Printing House for 
the Blind: Louisville, KY. Available from: 
http: //www. a ph. orq/tests/access/access. pdf 

Response Accommodations 

Students with visual impairments who use the presentation accommodations 
discussed above may also need to use certain response accommodations so 
that answers can be recorded appropriately. As with presentation 
accommodations, response accommodations with which the student is 
familiar are recommended. 

Considerations regarding response accommodations include the following: 

• The student may present answers orally to a test proctor who completes 
the answer sheet. 
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• Students may record onto audio tape answers that then must be 
transferred to the answer sheet. 

• The student may need to write answers in the test booklet or on 
separate paper. The student's answers will then need to be transferred 
to the answer sheet. 

• Students may use word processors to write answers that will be 
transferred to the answer sheet. Depending on the construct being 
tested, test administrators must verify that students are inhibited from 
accessing software or hardware that may provide an unfair advantage. 
For example, if a student is responding to a writing prompt and the 
writing will be judged based on correct spelling and grammar, then the 
spell check function and grammar functions must be disabled. 

Each of these accommodations requires a person to transfer the answers 
onto the scanable answer sheet or booklet that will be scored. If computer- 
based testing is used, the transfer of answers is not necessary as this 
process happens as part of the computer test program. The transfer of 
answers must be performed carefully to ensure that the student's answers 
are recorded as intended. 

The following guidelines are provided to ensure that this transfer of 
information is performed appropriately: 

1. Testing materials and the student responses are secure and confidential 
materials, and they must be treated as such to ensure test validity and 
the non-disclosure of the student's identity to unauthorized persons. 

2. Response transcribers must know braille if transcribing braille responses. 

3. Ideally, the response transcriber should be a "neutral" person, not 
someone with a vested interest in the student's scores. 

4. Response transcribers must record the student's use of punctuation, 
spelling, and grammar structure, and provide the student's answer 
exactly as it was delivered by the student. The response transcriber 
cannot record speculative responses for items that the student failed to 
complete. 

5. A second person should be made available to proofread the work of the 
response transcriber in order to ensure that the student's answers have 
been recorded accurately. For the same reason, two transcribers should 
work together in transferring to the answer sheet those graphics that 
the student has produced as an answer to a test item. 

6. For a period of time, student responses must be maintained in a secure 
file with test name, copyright year, form and level administered so that 
the student's actual responses can be reviewed if questions arise. 
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Setting Accommodations 

Frequently, students with visual impairments will need to take a test 
individually or in small groups to ensure that test accommodations are 
implemented without interference to the concentration and test taking of 
other students. If a student is being read to, is recording answers by using 
technology that is noisy, or is recording answers orally, then the student 
must take the test individually and under the supervision of a test 
administrator to prevent the distraction of other test takers. 

The setting for the testing situation must allow space for the materials to be 
used by the student. The manipulation of braille, large print materials, 
braillewriters, talking calculators, and large print materials requires that the 
student be allowed access to a flat, fairly large work area. Moreover, proper 
lighting, while sometimes overlooked, is critical for many readers with visual 
impairments. Lighting that has been adjusted to suit the student's particular 
visual needs will help promote sustained reading efficiency. 

Scheduling Accommodations 

The use of extended time for test completion is a testing accommodation 
that has received considerable attention since state testing and 
accountability systems have been implemented. Research investigating the 
use of extended time has yielded no conclusive information about its benefit 
(Tindal & Fuchs, 1999). However, students with visual impairments will 
usually require extended time during testing because using braille, large 
print, and audio format require more time than does reading standard print 
with acceptable visual acuity. 

A study by Gompel, van Bon, and Schreuder (2004) found that students with 
low vision can read effectively with their low vision aids, using 1 Vi to 2 times 
that needed by regular students. Traditionally, extended time for testing 
large print readers has been 1 Vi time, and for braille readers time allotted 
has been twice as much as that allowed for the standard print reader. 
Another study suggests that experienced braille readers may need no more 
that 50% additional time than the stated duration, with additional time 
allowed for the manipulation of an audio device or the marking of an answer 
sheet (Wetzel & Knowlton 2000). 

Regardless of the time allowed, the student should be carefully monitored to 
ensure that time is being used appropriately. If students need an inordinate 
amount of time, educators may need to investigate the efficiency of the 
chosen reading mode or initiate remediation to improve speed. Generally, 
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timing accommodations should be individualized according to the test taker's 
reading rate and testing situation (Wetzel & Knowlton, 2000). 

Reading braille or large print and listening to material presented orally, 
especially when accompanied by graphic material, can be a fatiguing and 
often frustrating experience in a high stakes testing environment. Therefore, 
students may need several brief sessions in which to take the test. Additional 
break options should also be considered. 

Students may need to be tested at different times of the day depending on 
their optimal functioning time. Students may also need to be tested over a 
longer time period, a week rather than two days, for example. However, any 
alteration of the timetable will necessitate close supervision to ensure test 
security. 


Special Tools Accommodations 

There are a number of special tools that students with visual impairments 
may need during the testing process. Tools provided for sighted students 
during testing, such as calculators, rulers, protractors, or other measurement 
devices, must be provided for students with visual impairments, as well. 
Talking calculators, braille or large print rulers, protractors, and other 
measurement devices do exist, and the student should be allowed to use 
them. When testing allows the use of non-scientific or scientific calculators, 
students with visual impairments should be permitted to use an equivalent 
device that has been adapted for use by the visually impaired user, e.g., a 
non-scientific or scientific talking calculator. Should a state provide 
calculators for the sighted population taking the test, then talking calculators 
should be provided to students with visual impairments who are taking the 
test. Before they are used in a testing situation, electronic and battery- 
operated devices should be inspected to ensure they function properly and 
that the devices contain no saved information, which might provide the user 
an unfair advantage. 

Some other special tools that students with visual impairments might use 
include: 

Abacus: An abacus is often useful for students when mathematics 
problems are to be calculated without a calculator. The abacus functions as 
paper and pencil for some students with visual impairments. 

Graphic Tools: If students are required to produce graphic 
information on a test, they should be allowed to use one of several graphic 
tool kits that exist. It is best if the student uses whatever method he or she 
has used during classroom instruction of graphic construction. The student's 
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constructed graph, if done in braille, will need to be transcribed into print for 
scoring. 

Line Markers and Templates: Occasionally students may want to 
use manipulative devices, such as a ruler or template, to maintain placement 
on a line of braille or print. 

Magnification Devices: Magnification devices include eyeglass- 
mounted magnifiers, free standing or handheld magnifiers, and electronic 
equipment such as the closed circuit television (CCTV) or a computer that 
has text enlargement software installed. These devices do not provide a 
student with an unfair advantage. Rather, they are devices that the student 
requires to access print, and they should be allowed as standard 
accommodations. Should a computer be used as an accommodation, the test 
administrator must ensure that only allowable computer options, such as 
screen enlargement, are used. 

Scientific Tables: Frequently, students may need to refer to a braille 
or large print edition of a scientific table, such as the periodic table of 
elements. 

Physical Manipulatives: Some testing situations may allow that 
objects presented on paper (i.e. money, geometric solids) can be substituted 
with a physical representation of the picture (i.e. penny, nickel, dime, 
quarter, or geometric solids used in instruction). 

Special Paper: Students may need specially designed bold line or 
raised line paper for constructing answers and producing graphs. 

Summary 

This paper has outlined the typical accommodations used by students with 
visual impairments when being tested through use of a written assessment 
such as an academic achievement test. While this discussion is not 
exhaustive of all accommodations that might be used, it is intended to 
provide an understanding of the general accommodations that are expected 
when assessing a student with a visual impairment. Documentation of these 
accommodations on the IEP is crucial as is routine evaluation of their 
effectiveness. 
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APPENDIX G 

Educational Testing Service (ETS) 
GUIDELINES FOR A TEST READER 

The following guidelines will assist in providing the testing 
accommodation of a reader for a test taker with disabilities. If you have 
questions about a specific test, please contact a testing program 
representative. 


Characteristics of a Good Reader 

1. Ability to read aloud clearly, at a normal pace, and with good 
pronunciation. 

2. Familiarity with the words, terms, symbols, or signs that are 
specific to the test content. 

3. Ability to follow instructions to read, verbatim, only the words in 
the test book or on the screen, without changing or adding words 
or assisting the test taker in selecting a response. 

4. Willingness to be patient and to understand that the test taker 
may need to have many test questions repeated several times. 

5. Ability to work with the test taker comfortably and compatibly 
without creating unnecessary pressure or unrealistic expectations. 

General Information for Readers 

1. You must review the test format, subject matter, and sample test 
questions in the testing program's information bulletin or by 
visiting the testing program's Web site. 

2. Prior to beginning the test, you will have the opportunity to meet 
with the test taker, who should be encouraged to discuss matters 
that will affect test performance, e.g., how to determine the 
amount of remaining time and how you can help pace the test taker 
through the test. The opportunity to discuss such questions and 
concerns before the test administration begins will make the test 
administration more effective and fair and will help to minimize 
misunderstandings and misinterpretations. 

3. Test takers who are blind or who have low vision may also have 
special tools or equipment (e.g., abacus, brailler, slate, and stylus) 
that have been approved for use during the test. These tools offer 
neither an unfair nor a special advantage; they are comparable to 
paper and pencil and accomplish the same task. The most 
important consideration is for you and the test taker to have the 
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same set of expectations about what is to happen, how much time 
is allowed, and how all the tasks will be accomplished. 

4. Test takers who are blind or who have low vision may also have 
special tools or equipment (e.g., abacus, brailler, slate, and stylus) 
that have been approved for use during the test. These tools offer 
neither an unfair nor a special advantage; they are comparable to 
paper and pencil and accomplish the same task. The most 
important consideration is for you and the test taker to have the 
same set of expectations about what is to happen, how much time 
is allowed, and how all the tasks will be accomplished. 

5. The test taker may require all or portions of the test to be read 
aloud. The test taker depends on the reader to read the test 
questions accurately, to pronounce words correctly, and to speak 
in a clear voice throughout the test, which may go on for several 
hours. It is a demanding and somewhat tedious task, and not 
everyone is suited to do it. Drinking water should be available for 
you. 

6. Your task is to read only the test questions. Do not try to solve 
problems or determine the correct answer as you read because 
this may result in an unconscious pause or change in inflection 
that could be misleading or disconcerting to the test taker. The 
expression on your face should remain neutral. Do not look at the 
test taker or smile or frown to indicate approval or disapproval. 

7. Read each question as clearly as possible. Give special emphasis 

to words printed in boldface, italics, or capitals, and tell the test- 
taker that the words are printed that way. Do not give your own 
emphasis to words not emphasized in print. 

8. If you find an unfamiliar word or one that you are not sure how to 
pronounce, advise the test taker of your uncertainty about the word 
and spell it. 

9. When reading a word that is pronounced like another word with a 
different spelling, if there can be any doubt about which word is 
intended, spell the word after you have pronounced it. Spell any 
words requested by the test taker. 

10. Avoid getting into conversation about the test questions, but try to 
respond to the test taker's questions by repeating the item, words, 
or instructions as needed. 
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11. When reading passages, be alert to all punctuation marks. Read 
the passage through once so that the test taker can grasp the 
content of the passage. Some test takers may ask for the passage to 
be read through a second time with punctuation marks indicated. 
When required or asked to read, with punctuation, specific lines 
within a passage, indicate all punctuation found within those lines. 

12. When test questions refer to particular lines of a passage, reread 
the lines before reading the question and answer choices. For 
example, you might say, "Question X refers to the following 
lines..." Reading the lines referred to would then be followed by 
reading question X and its response options. 

Special Considerations for Multiple-Choice Tests 

1. Be particularly careful to give equal stress to each response option 
and to read all of them before waiting for a response. The test- 
taker will record the answer or provide the answer to the test 
administrator (writer), who will record it for the test taker. 

2. If you are recording answers and if the test taker designates a 
response choice by letter only ("D", for example), ask if you 
should reread the complete response before the answer is 
recorded. 

3. If the test taker chooses an answer before you have read all the 
answer choices, ask if you should read the other response options. 

4. Allow the test taker to pause before responding. However, if the 
test taker pauses for a considerable time following your reading of 
the answer choices, say: "Do you want me to read the question 
again...or any part of it?" In rereading questions, be careful to 
avoid any special emphasis on words not emphasized in the 
printed copy by italics or capitals. 

Mathematics Reading 

A test taker is permitted to ask the reader to write notes and to assist 
with intermediate steps in computing mathematics problems, especially 
if the test taker has no tools or equipment for taking notes or is unable 
to do so. For example, in the multiplication of numbers (e.g., 17 x 521), 
a test taker may say, "Seven times one is seven. Put down the seven. 
Seven twos are fourteen. Put down the four to the left of the seven and 
carry the one." The test taker should be specific in directions to the 
reader as to what he or she writes, in which column to write it, what to 
carry, etc. 

Mathematical expressions must be read precisely and with care to avoid 
misrepresentation for a test taker who has no visual reference. For math 
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items involving algebraic expressions or other mathematical notation, it 
may be preferable for the reader to silently read the entire question 
before reading it aloud to the test taker. Use technically correct yet 
simple terms, and be consistent in the treatment of similar expressions. 
Some typical expressions and the manner in which they should be read 
follow: 

(a) Lowercase letters that are juxtaposed should be read as a 
multiplication expression: 

e.g., xy should be read as "x y," unless it is part of a complex 
expression or this reading is otherwise unclear, in which case read 
it as "x times y." 

(b) Capital and lower-case letters should be differentiated because 
they can have different meanings in mathematical or scientific 
expressions: 

e.g., 


R - 2y = 6 

should be read as "Capital R minus two y equals six." 

(c) Simple numerical fractions should be read as fractions: 
e.g., 


5/6 

Should be read as "five sixths." 

(d) However, similar letter expressions can be read as one letter 
"over" another: 

e.g., 

a 


b 

Should be read as "a over b." 

b+d 


c 

Should be re ad as "a fraction with numerator b plus d and 
denominator c." 

If there is any question as to where the fraction ends, say "end fraction." 

(e) Negative numbers should be read as "negative": 
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e.g. f 


-5 

should be read as "negative five," not "minus five." 

When a subtraction operation is involved, read the sign as "minus," 
e.g. : 


x - 5 

should be read as "x minus five." 

(f) Expressions containing multiple mathematical operations should 
be read exactly as they appear. Expressions containing 
parentheses or brackets can be read in any of the following three 
ways: 


1. quantity, close quantity 

2. paren, close paren (or bracket, close bracket) 

3. left paren, right paren (or left bracket, right bracket) 

For "paren, close paren" or "left paren, right paren," it is also 
acceptable to use "parenthesis" instead of "paren." 

If you use the term "quantity," in complicated expressions, announce 
where enclosed portions end by saying "end quantity:" 


(2x - 6y) - 10 


could be read 

• As "The quantity two x minus six y, close quantity, minus ten;" 

• As "paren, two x minus six y, close paren, minus ten;" 

• Or as "left paren, two x minus six y, right paren, minus ten." 

a (x - y) 

could be read as "a, parenthesis, minus y, close parenthesis." 
a x b 2 
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could be read as "a times the square of b." 

Use pauses to audibly group sections of an expression together: 

e.g.. 


z + (-a) 

could be read as "z plus [PAUSE] paren negative a close paren." 

(g) If equations are used in the test you will be reading: 

Since equations are a shorthand means of stating relationships between 
quantities, the reader's job is to translate this shorthand back into 
everyday English. Read equations in this order: 

1. If the equation is numbered, read its number first. 

2. Give the meaning of each letter or symbol 

3. Read the equation. 

e.g.: 

Eq. 6-2 

E = energy in ergs 
m = mass in grams 
c = speed of light in cm./sec. 

E = me 2 

Read as "Equation six dash two. Capital E equals energy in ergs, m 
equals mass in grams, and c equals the speed of light in 
centimeters per second. Then, Capital E equals m c squared." 

Test Center Procedures for Using a Reader 

1. An approved reader should be admitted to the test center with the 
test taker. The reader's photo-bearing identification should be 
checked. 

2. Prior to the start of the exam, the test center administrator/ 
supervisor will review the Guidelines with the test taker and the 
reader and will set the ground rules for the conduct of the 
examination. 

3. The test administrator must remain in attendance at all times 
during the test administration. 

4. An approved reader is not present to function as an aide to the 
test center staff. It is inappropriate to ask the reader to perform 
clerical duties of any kind. The reader should not be asked to 
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assume any responsibilities belonging to either the center staff or 
the test taker. 

5. Test center staff must ensure that proper test security is maintained 
at all times. It is important that the test administrator ask 
questions and avoid any hasty interpretations of what may be 
communication of test content or exchange of information between 
the test taker and the reader that might give the test-taker an 
unfair advantage. The task requested by the test taker might be 
acceptable once understood. Discussion or communication concerning 
interpretation of test content is not permitted. If such discussion 
occurs and cannot be controlled, or if test center staff observe 
anything they deem unusual, the situation should be reported on the 
Supervisor's Irregularity Report (SIR) or the Electronic Irregularity 
Report (EIR) and the test taker advised of this action. 

6. The test center administrator may also stop the test and dismiss 
the test taker if he or she believes that the reader has provided 
the test taker with any unfair advantage. In such instances, ETS 
reserves the right to cancel the test taker's score. 

References 

Educational Testing Service. (2003). ETS Guidelines for a Test Reader. 
Retrieved August 1, 2008, from 

http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20b 

c47c3921509/?vgnextoid=7889bc914be45010VgnVCM10000022f95190 

RCRD&vgnextchannel=d7f7be3a864f4010VgnVCM10000022f95190RCR 

D 

ETS materials selected from ETS Guidelines for a Test Reader , 2003. 
Reprinted by permission of Educational Testing Service, the 
copyright owner. For limited use by the American Printing House for 
the Blind, Inc. 
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JA3233 



Case l:14-cv-00857-TSC Document 70-50 
USCA Case #17-7035 Document #1715850 


Filed 01/22/16 Page 121 of 127 
Filed: 01/31/2018 Page 212 of 441 


Exhibit D 


JA3234 



Building Assessment Initiatives for Schools: Guidelines to Support the Contract Development Proce... Page 1 of 6 
Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 122 of 127 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 213 of 441 


A p [J American Foundation® 
r\r D for the Blind 


Expanding possibilities for people with 
vision loss 


Building Assessment Initiatives for 
Schools: Guidelines to Support the 
Contract Development Process Between 
Test Publishers and States 


Braille Downloads 

Download .BRF version of guidelines for contract development 

Download .DXB version of guidelines for contract development 

19th Annual Josephine L. Taylor Leadership Institute 

Boston, Massachusetts 
Friday, March 11,2005 

Introduction 

Contracts and Requests for Proposals (RFPs) negotiated between state assessment agencies 
and test publishers carefully outline the responsibilities and expectations for the state 
assessment development and implementation process. While these documents have specific 
points for consideration, often language does not include the assurance of accessible test 
development and implementation. Accessible test items enable all students to participate in the 
assessment process in a way that allows abilities rather than disabilities to be assessed. 
Accessible formats of tests, including the practice tests, must be available for students with visual 
impairments at the same time as their sighted peers. The checklists provided below outline 
considerations for inclusion in each state's RFP or test contract. The usual contractual language 
found in state contracts should be employed, with these special considerations added. 

Universal Design Principles 
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The following guidelines are general considerations for contract and RFP development that 
ensure test development and use for all students, including those with disabilities: 

The same assessment system is used to measure the achievement of all public school 
students in the state. Groups to be included in the state assessment need to be clearly defined. 

The student assessment system provides coherent information on attainment of state 
standards across grades and subjects. 

The tests are designed to be valid and accessible for all students. This includes students with 
disabilities and students with limited English proficiency. 

The tests are aligned with the state's challenging academic content and student achievement 
standards. 

The tests are valid, reliable, technically sound, and consistent with nationally recognized 
professional and technical standards such as national test publisher standards and guidelines 
of the American Psychological Association (APA) and American Educational Research 
Association (AERA). 

The reporting system allows results to be disaggregated (according to the No Child Left Behind 
Act (NCLB) and Individuals with Disabilities Improvement Act of 2004 (IDEA) guidelines) within 
each state and local education agency and school by gender, racial and ethnic group, migrant 
status, disability, socioeconomic status, and limited English proficiency. 

The tests involve multiple up-to-date measures of student academic achievement, including 
measures that assess higher-order thinking skills and understanding. 

The reporting system allows production of individual student reports. 

Roeber, E. (2003). Assessment models for No Child Left Behind. 

Education Commission of the States. http://www.ecs.org/html/Document.asp?chouseid=4009 

Item Development and Review Process with Publisher 

The following guidelines are provided for consideration as language to include in contracts and 
RFPs that ensure the development and implementation of accessible test formats, specifically for 
students with visual impairments. 

Test publishers must maintain access to experts, i.e. individuals who know and have either 
taught or are knowledgeable about braille, tactile graphics, large print, and audio. These 
individuals can provide information during each phase of test development. 

Experts in visual impairment must be included on Item Writing Committees and Bias Review 
Committees. 
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The use of accommodations must be considered during test item development to ensure 
appropriateness to test purpose and test access. 

The test item pool must be large enough for Bias and Item Review Committees to replace items 
determined to be inaccessible when presented in braille, large print, audio formats, or as tactile 
graphics. 

A representative sample of students with visual impairments needs to be included in any field- 
testing of the assessments, as prescribed in Standard 10.3 (p. 106) of the Standards for 
Educational and Psychological Testing (1999). 

An adequate amount of time for tests and practice tests to proceed through a subcontractor's 
processes needs to be built into contracts so that accessible media as required by each 
student's Individualized Education Program (IEP) are delivered at the same time as the original 
test materials. 

All test administrators' manuals, supplemental manuals which accompany the accessible media 
versions of tests, and local test administrators'/proctors' instructions and training manuals must 
be provided in accessible formats for visually impaired staff. These accessible materials must 
be requested far enough in advance to allow for delivery at the same time as the original test 
materials. 

At the end of each testing season, both students and teachers should give input regarding the 
testing experience. 

Item analyses for accessible format test items will be carried out at the end of each school year 
(or testing season) as part of a continuous improvement plan. 

Allman, C.B. (2004). Test Access: Making tests accessible for students with visual impairments: 

A guide for test publishers, test developers, and state assessment personnel. Second Edition. 
Louisville, KY: American Printing House for the Blind. 
http://www.aph.org/tests/access2/index.html 
Accessible Media Development with Subcontractors 

This section provides guidelines for consideration when contracts are developed with 
subcontractors such as agencies or individuals who will provide tests in one or more accessible 
formats (braille, tactile graphics, large print, and audio). The process may include steps for 
editing, transcribing, designing tactile graphics, proofing, producing and quality checking the 
accessible media. It is essential that the timeline allow adequate time for each of these steps. 

Additional time may need to be built into contracts depending on specific requirements of the 
state such as an independent proofreading by another person or agency, or aligning various 
media for multimedia presentations. 
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The subcontractor must agree to work closely with the test publisher, the state department of 
education, and the test editor. 

The construct to be measured must be documented by the test publisher in test item 
specifications and made available to test editors and accessible media producers. 

Proofreading by a qualified individual, i.e. a person who knows the needed codes and formats 
and is experienced or certified (if applicable), in braille, tactile graphics, large print, and audio 
versions of the test must occur before multiple copies are made. High-stakes tests should be 
proofed a minimum of two times. 

Accessible versions of the test must be aligned so that a multimedia presentation (as approved 
by state assessment programs) is possible if specified by a student's IEP. 

Allowable test format changes, accommodations, and general assistance to test takers by the 
test administrator or proctor must be stated in the test administration manual or supplemental 
materials produced by the subcontractor. 

Subcontractors must be able to meet their deadlines so that high quality accessible media are 
delivered to school systems at the same time as the original test materials. 

Test security and confidentiality standards must be upheld by testing subcontractors. 

Allman, C.B. (2004). Test Access: Making tests accessible for students with visual impairments: 

A guide for test publishers, test developers, and state assessment personnel. Second Edition. 
Louisville, KY: American Printing House for the Blind. 
http://www.aph.org/tests/access2/index.html 

Resources 

Assessment Models for No Child Left Behind, 
from Education Commission of the States (ECS) 
http://www.ecs.orq/clearinqhouse/40/09/4009.doc 

Building Tests to Support Instruction And Accountability: A Guide for Policymakers, 

from National Education Association (NEA) http://www.nea.org/accountability/buildingtests.html 

Designing School Accountability Systems, 

from Council of Chief State School Officers (CCSSO) 

http://www.nciea.org/publications/desiqSchAccSvst Gonq02.pdf 

Illustrative Language for an RFP to Build Tests to Support Instruction and Accountability, 
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from American Association of School Administrators (AASA) 

http://www.aasa.org/issues and insights/assessment/lllustrative Language for an RFP.pdf 

Information on Writing a Request for Proposal (RFP) 
http://www.arches.uga.edu/~ninaaug/ITclasses/7550/ 

Model Contractor Standards and State Responsibilities for State Testing Programs, 
from Education Leader's Council (ELC) http://www.accountabilityworks.org/publications 

National Federation of the Blind (NFB/New Hampshire Resolution on Accountability) 
http://www.education-rights.org/brailletwomey11399.html 

Tennessee RFP for Development of Online Tests 
http://www.state.tn.us/finance/rds/ocr/rfp/rfp33104001 .pdf 

Test Access: Making Tests Accessible for Students with Visual Impairments: 

A Guide for Test Publishers, Test Developers, and State Assessment Personnel. Second Edition. 
American Printing House for the Blind, http://www.aph.org/tests/access2/index.html 

The Standards for Educational and Psychological Testing (1999), 

American Psychological Association http://www.apa.org/science/standards.html 

Thompson, S., & Thurlow, M. (2002). Universally designed assessments: Better tests for 
everyone! (Policy Directions No. 14). 

Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. 

Retrieved 1-28-05 from the World Wide Web: 
http://education.umn.edu/NCEO/OnlinePubs/Policv14.htm 

Contributors 

Dr. Carol Allman, Accessible Tests Department with the American Printing House for the Blind, 
allmanc@prodigy.net 

Barbara Henderson, Accessible Tests Department with the American Printing House for the 
Blind, bhenderson@aph.org 

Debra Sewell, Texas School for the Blind and Visually Impaired, debrasewell@tsbvi.edu 
Mary Ann Siller, American Foundation for the Blind, siller@afb.net 


JA3239 

http://www.afb.org/info/afb-national-education-program/jltli-2005-education-summary/cliecklist-for-rlp-buildin... 














Building Assessment Initiatives for Schools: Guidelines to Support the Contract Development Proce... Page 6 of 6 
Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 127 of 127 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 218 of 441 

Debbie Willis, Accessible Tests Department with the American Printing House for the Blind, 
dwillis@aph.org 

Permission is given to distribute copies with appropriate credit: American Foundation for 
the Blind, American Printing House for the Blind and Texas School for the Blind and 
Visually Impaired from the Josephine L. Taylor Leadership Institute, March 11, 2005. 


Copyright© 2015 American Foundation for the Blind. All rights reserved. 
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OCR Issues Draft Guide on 
Disparate Impact in Educational Testing 


Access 


What is l-O? 

Industrial-organizational (l-O) 
psychology is the scientific study of 
the workplace. Rigor and methods of 
psychology are applied to issues of 
critical relevance to business, 
including talent management, 
coaching, assessment, selection, 
training, organizational development, 
performance, and work-life balance. 



El 0 B3 


Wayne Camara 
The College Board 

In May, the Department of Education’s Office of Civil Rights (OCR) released a draft Resource 
Discrimination in High Stakes Testing" that sought to provide an overview of federal standard; 
principles that should guide the use of tests for making high stakes educational decisions (e.c 
special educational referrals, promotion, graduation, and scholarship awards). This Resource 
development for several years according to OCR, but educational groups and test publishers 
working days before it was originally scheduled for release. 

The Guide may have limited direct impact on 1-0 psychologists, unless they are involved in ec 
However, the Guide may be of interest for other reasons, since it interprets and applies both 
employment arena and professional testing standards to issues of disparate impact in ways t 
"overreaching" or incorrect. 

Test publishers, APA, and other educational institutions objected to the proposed timing of the 
agreed to revise the current document with plans for a fall publication. OCR has stated the G 
new federal guidelines or professional standards, but rather will provide a meaningful interpre 
tests in education. A number of national media outlets (New York Times, Wall Street Journal, 
Chronicle of Higher Education) have run stories on the guidelines and op-ed pieces that have 
emphasis on disparate impact being the sole determination of whether or not a test should be 

The Guide cites specific wording from the Standards for Educational and Psychological Testir, 
40 occasions, leading APA, AERA, and NCME to formally request that OCR delay revision of 
has been revised and published (sometime around December 1999). Several organizations h 
comments on the OCR Guide. 

The Guide attempts to apply Title VII law, EEOC Guidelines, and professional standards that 
to educational test use. It cites several Supreme Court and lower courts decisions concerning 
or transports decisions and standards to education. Major concerns addressed by educations 
summarized in comments submitted by the College Board (Camara, 6/21/1999): 


First, the Resource Guide focuses exclusively on disparate impact resulting from tests (or diff 
ignores the level of validity and utility offered by a test. Disparate impact cannot be considere 
must be evaluated in terms of the overall validity and utility of inferences associated with the 
Resource Guide clearly elevates any measure, irrespective of validity, cost, or burden to the < 
lower disparate outcomes above any test having greater disparate outcomes. We believe this 
precedent that has no legal or professional justification and the Guide will have a chilling effe 
educational tests. 


Second, the Resource Guide offers no guidance on what level of disparate impact would rest 
there be substantial statistical disparities or would any disparate outcome result in an investig 
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IBSSa statisticaF^^y^gg^^^ermiip^^^^qgif alternative me 
A consistent pattern of ethnic and racial disparities has been found across a variety of standa 
National Assessment of Educational Progress (NAEP) and the National Educational Longitud 
educational measures used for high-stakes decisions, such as high school grades, class ranh 
quality and rigor of courses completed, as well as educational outcomes (e.g., college grades 
(Camara and Schmidt, under review). Disparities in test results reflect similar differences in o 
(e.g., job performance, college achievement, and grades) and may be indicative of earlier diff 
learn and educational opportunities, not test bias or flaws with the test. 


Third, professional and technical standards do not define tests so narrowly that they exclude 
assessments that are both used daily to make high-stakes decisions about individual student 
have similar levels of disparate impact against protected groups. Specifically, the Test Stand; 
standardized ability (aptitude and achievement) instruments, diagnostic and evaluative device 
personality inventories, and projective instrumentsa more appropriate choice among assessn 
use will be facilitated if there is a reasonable comparability in the kinds of information availabl 
three broad categories of test instruments are covered [emphasis added]: constructed perfor 
and to a lesser extent, structured behavioral samples (pages, 3_4)." Related to this comment 
Resource Guide be renamed to put added emphasis on Measures Used in Making High-Stak 
Uniform Guidelines and Employment Selection Procedures), rather than focus exclusively on 
decision-making process, testing. 


Fourth, we applaud OCR's deference to the Test Standards. However, the Resource Guide ir 
professional standards can be applied in a rigid manner in evaluating tests. The Test Standar 
rigid checklist approach, noting that specific circumstances affect the relevance of standards 
must be applied in evaluating tests. Professional practice and standards are typically constru 
other measures need not meet all standards to be appropriately used within the bounds of pr 
(Richardson, 729 F. Supp. At 821,823). In addition, the three sponsoring educational associ; 
the Standards, which date back to 1985. We strongly endorse the recommendations from AP 
asking that issuance of this Resource Guide be deferred until after publication and dissemina 
Standards and requesting a standard 90-day review period for any subsequent drafts of this c 
publication of the revised Test Standards. 


Fifth, we would ask OCR to ensure that colleges and universities, school districts, and state e 
an opportunity to review and comment on this proposed Resource Guide. The Resource Guic 
disseminated or reviewed by colleges and secondary schools. These are the very organizatio 
affected by the Resource Guide once it is issued and it seems appropriate that they be given 
comment on the inferences and proposed standards. 


Sixth, the distinction the Resource Guide makes between tests and other assessment device 
establishing a much lower technical, professional, and legal standard for more subjective ass 
applications, grades and GPA, recommendations, ratings or evaluations of student work and 
experiences and honors, community service and involvement, samples of student work). In l/l 
and Trust, 487 U.S. 977, the American Psychological Association submitted an amicus curia; 
argued there is no professional or scientific justification to treat subjective and objective devic 
validation requirements. In fact, not imposing essentially the same legal and technical standa 
and devices used in high-stakes individual decisions would provide a sanctioned and covert r 
APA further argued that subjective procedures (in that case used for employment) are "amen 
psychometric scrutiny" as objective procedures, citing the Test Standards which address inte 
(Camara, 1996). In deciding Watson v. Fort Worth Bank and Trust, 487 U.S. 977, all eight of 
O'Connor's opinion holding that the adverse impact theory can be used in cases involving sut 
was concerned that an employer could combine an objective criteria (such as a test or diplorr 
(such as interviews or ratings) and easily insulate itself from the Griggs test. O'Connor noted 
making systems could have "precisely the same effects as a system pervaded by impermissit 
(Opinion at 4926). 


Seventh, professional and legal standards do not provide any support for OCR's distinctions I 
and other measures. We agree with comments to an earlier draft of this Resource Guide sub 
Testing and Assessment (Shavelson, June 10, 1996), stating that "OCR's inquiry is not to pro 
validity of inferences and decisions based on tests, but rather to determine whether the entire 
a part provides students a fair and equal opportunity to learn...." The Resource Guide ignores 
even if they contribute more to disparate outcomes. In fact, high school courses, judgments a 
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testing, if an institution places substantially greater weight on these factors. For example, if te 
one of several factors in admissions, then there is no guarantee those disparate outcomes wi 
eliminated. In requiring tests to meet an exceptionally higher standard than other measures (< 
student work, high school rank, past experience, and opportunities), the Resource Guide will 
of valid and objective standardized tests used by educational institutions, states, and school c 
educational institutions may opt to employ less valid and less objective methods for high-stak 
are not addressed in this Resource Guide. 


Eighth, the Resource Guide also sanctions the use of the Uniform Guidelines on Employment 
resource in educational testing. As the Resource Guide acknowledges in a footnote, there ar< 
differences between educational and employment testing that we believe undermines any att 
in educational settings. The Uniform Guidelines were never developed with application to edi 
organizations did not have an opportunity to comment on extensions of the principles to educ 
Guidelines are over 25 years old and do not reflect current scientific principles of measureme 
practice. The Uniform Guidelines are outdated and do not conform to the Testing Standards ( 
their consideration of validity (as accomplished by adopting one of three distinct types of valic 
(this is virtually ignored in the Guidelines, but is accepted professional practice), differential p 
as well as several other areas (APA, 1985). The Uniform Guidelines may provide a framewor 
guidelines addressing test use, but they should not be viewed as a substantive resource in ec 

Ninth, statistical analyses should be based on the pool of qualified applicants, not a general p 
not addressed in the Resource Guide. 


Tenth, this Resource Guide implies that once disparate impact is established that the burden 
educational institution to demonstrate both the educational necessity of the test and then to d 
alternative exists throughout the process. This legal interpretation is incorrect. 

Other sections of the Resource Guide viewed as problematic include wording implying that se 
studies are required for each school; that tests can only be used for purposes they were origi 
than for uses where sufficient validation evidence exist); and that there is a unique methodolc 
when they are to be used as the sole criteria. 

On June 18 th , the House held a hearing on the OCR Guide and department officials noted tha 
recirculate the current draft to groups who have already submitted comments on the current c 
submit a revised Guide to the National Academy of Sciences Board of Testing and Assessme 
Thereafter, they anticipate making a final draft available for public review this fall. They will pi 
and will have the revised Guide posted on their web (Coleman, June 21, 1999, personal com 
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IT you work with he Federal 
Register (FR) or the Code 
of Federal Regulations 
(CFR), you may find hese 
free workshops especially 
valuable. 

You can also take the: 
On-line Tutorial 


Leam why 
Democracy Starts 
Here 



Who to Contact 

. . About IBR 


For more information about a standard: 


1. Use the contact information contained in the 
regulation to: 

* Contact the agency that issued the 
regulation containing the IBR standard. 


Incorporation by reference (IBR) allows Federal 
agencies to comply with the requirement to 
publish rules in the Federal Register and the 
Code of Federal Regulations (CFR) by referring 
to materials already published elsewhere. 

Learn More 


♦ Contact the standards organization that 
developed and published the material. 

Some standards organizations have online reading rooms that are free to the public, to 
registered users, or to organization members. Some of the standards incorporated by 
reference may be accessible at these standards organization web sites: 


• ASTM International free online reading room 

• ASHRAE free resources 

• NFPA free access to codes and standards 

• ANSI incorporated by reference (IBR) portal 

• Underwriters Laboratories standards incorporated by reference 

• International Code Council (ICC) free resources 

• Manufacturers Standardization Society (MSS) reading room 


contact aircraft and aircraft parts manufacturers directly. 

Some service information incorporated by reference in airworthiness directives may be 
available online. 


2 You can also find agency phone numbers and other contact information at: 

• USA.gov 

• United States Goverment Manual 

• Federal Citizen Information Center, National Contact Center 

3. You may also use the NIST database, Regulatory Standards Incorporated by Reference, for 
information on the availability of IBR standards. 
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Generally, members of the public must pay a fee to receive a copy of the incorporated material. 
If you have difficulty locating the material, contact the regulatory agency that issued the 
regulation. 


Why is Incorporation by Reference Used? 


Incorporation by reference is used primarily to make privately developed technical standards Federally 
enforceable. Agency generated documents are presumptively ineligible for incorporation by reference 
because that material can and should be published in full text in the Federal Register and CFR. 
Agencies are not authorized to incorporate by reference material on their web sites as a substitute for 
Federal Register publication. 

The legal effect of incorporation by reference is that the material is treated as if it were published in the 
Federal Register and CFR. This material, like any other properly issued rule, has the force and effect 
of law. Congress authorized incorporation by reference in the Freedom of Information Act to reduce 
the volume of material published in the Federal Register and CFR. ( See 5 U.S.C. 552(a) and 1 CFR 
part 51). Congress gave complete authority to the Director of the Federal Register to determine 
whether a proposed incorporation by reference serves the public interest. 

P Tnn nf Page | 

Where to Find Materials Incorporated by Reference at NARA Facilities 


In most cases, materials incorporated by reference are made available through the standards 
organization that developed the standard. Contact the standards organization or other designated 
sources through the address listed in the Federal Register or CFR. 

However, legal record copies of material incorporated by reference are also filed at the Office of the 
Federal Register (OFR) and other NARA facilities. OFR does not distribute IBR materials. 

Legal record copies are available for public inspection and limited photo-copying. If you would like to 
inspect material incorporated by reference at OFR's downtown Washington, DC location, you must 
submit a written request and make an appointment for a specific day and time. 

1. Submit your written request at least a day in advance. 

2 Your request must include: 

• Your name and daytime contact information—so we can confirm your appointment and the 
availability of the material you are seeking or in case we have questions, 

• A detailed description of the material you wish to examine, and 

• The date and time you wish to examine the materials. 

3. Submit your request by: 

E-mail fedreg.legal@nara.gov 

^ U.S. Mail addressed to: 

Office of the Federal Register (NF) 

The National Archives and Records Administration 

8601 Adelphi Road 

College Park, MD 20740-6001 

* Note that our mailing address differs from our physical location. 

If submitting your request by mail, we must receive your request at least a day in 
advance of your requested inspection date. 


http://www archives gov/federal-register/cfr/ibr-locations html[l/20/2016 2:42:44 PM] 
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the point that they are transferred from OFR to other NARA sites on a regular basis. See the 


Disposition Schedule below for more information on where materials are housed and use the links for 
these facilities to learn about researcher and information access policies at those locations. 


H lop o' Page] 


Disposition Schedule and Location 


The following table is a listing of the disposition schedule and location of the materials incorporated by 
reference: 

• The dates and timeframes are approximate 

• Addresses for each location are listed below the table 


Category of Records 

Location of Record 
Period 

OFR WNRC 

- Retention 

NARA 

Aircraft Service Bulletins for FAA Airworthiness Directives (14 CFR 39) 

Year 

0-3 

Year 

3-10 

From Year 

10 Forward 
(permanent 
storage) 

State Implementation Plans and Amendments submitted to EPA (40 CFR 
part 52) 

Year 

0-5 

Year 

5-15 

From Year 

15 Forward 
(permanent 
storage) 

All other materials incorporated by reference in the CFR 

Year 

0-5 

Year 

5-15 

From Year 

15 Forward 
(permanent 
storage) 
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Addresses 


These are the addresses of the locations listed in the table above. Please call 202-741-6030 for help 
in determining where the materials are housed: 

Office of the Federal Register (OFR) 

800 North Capitol Street NW, Suite 700 
Washington, DC 20001 

Washington National Records Center (WNRC) 

4205 Suitland Road 
Suitland, MD 20746-8001 

National Archives at College Park (NARA) 

8601 Adelphi Road 
College Park, MD 20740-6001 


If you are interested in obtaining a copy of a standard that 
has been incorporated by reference, contact the 
standards organization that developed the material or the 
agency that incorporated it. 


Contact the Standards 
Organization or Agency 


If you are interested in examining material that has been 
incorporated by reference, submit a written request to the 
Office of the Federal Register. 


For more information about Incorporation by Reference, 
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Telephone (202)741-6030 


Inspect IBR Materials at 
OFR 


^ Fax (202)741-6012 


E-mail fedreg.legal@nara.gov 


^ U.S. Mail addressed to: 

Office of the Federal Register (NF) 

The National Archives and Records Administration 
8601 Adelphi Road 


Federal Register > 


Information For... Publications 


Orgs & Offices 


I Want To... 


Resources 


Citizen Archivists 
Federal Employees 
Genealogists 
Members of Congress 
Preservation 
Records Managers 
The Press 


Federal Register 
Free Publications 
Prologue Magazine 
Purchase Publications 
More- 


Center for Legislative Archives 
Federal Records Genter 
Office of the Inspector General 
Presidential libraries 
More... 

About Us 

What is the National Archives? 
Doing Business with Us 
Plans and Reports 
Open Government 
Our Plain Language Activities 


Get My Military Record 
Plan a Research Visit 
Visit the Museum 
View Online Exhibits 
Apply for a Grant 
Participate 
Attend an Event 
Donate to the Archives 
Work at the Archives 
Volunteer at the Archives 


A-Z Index 

America's Founding Docs 
Contact Us 
EnEspanol 
FAQs 


Contact Us | Accessibility | Privacy Policy | Freedom of Information Act | No FEAR Act | USA.gov 
The U.S. National Archives and Records Administration 
1-86-NARA-NARA or 1-866-272-6272 
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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN EDUCATIONAL RESEARCH ) 

ASSOCIATION, INC., AMERICAN ) 

PSYCHOLOGICAL ASSOCIATION, INC., ) 

and NATIONAL COUNCIL ON ) 

MEASUREMENT IN EDUCATION, INC., ) 

) 

Plaintiffs/Counterclaim-Defendants, ) 

) 

v. ) 

) 

PUBLIC.RESOURCE.ORG, INC., ) 

) 

Defendant/Counterclaim-Plaintiff. ) 

_ ) 


Civil Action No. 1:14-cv-00857-TSC-DAR 

PLAINTIFFS’ REPLY IN FURTHER 
SUPPORT OF ITS MOTION FOR 
SUMMARY JUDGMENT AND 
PERMANENT INJUNCTION AND 
OPPOSITION TO DEFENDANT’S 
MOTION FOR SUMMARY 
JUDGMENT 


Respectfully submitted, 

Jonathan Hudis (DC Bar # 418872) 

Nikia L. Gray (pro hac vice ) 

Jonathan P. Labukas (DC Bar # 998662) 
QUARLES & BRADY LLP 
1700 K Street NW, Suite 825 
Washington, DC 20006-3825 
Tel. (202) 372-9600 
Fax (202) 372-9599 
E-Mail Jonathan.Hudis@quarles.com 
E-Mail Nikia .Gray@quarles.com 
E-Mail Jonathan.Labukas@quarles.com 

Counsel for Plaintiffs American Educational 
Research Association, Inc., American 
Psychological Association, Inc., and 
National Council on Measurement in 
Education, Inc. 
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Resource posted online and what those individuals have done with such copies (Dft. SDF at ]fl[ 
89-90 [Dkt. 68-3]; Plfs. SDF at ^ 51). Additionally, because Public Resource’s dissemination of 
the 1999 Standards does not provide notice that they have been replaced by the 2014 Standards, 
Plaintiffs expect a loss of revenue from sales of authorized copies of the 2014 Standards, and 
harm to the public due to the sale of outdated standards (Dft. SDF at | 99 [Dkt. 68-3]; Plfs. SDF 
at ][ 51; Geisinger Deck ]fl[ 25-29). Additionally, Plaintiffs presented evidence and case law 
concerning the irreparable injury that results from Plaintiffs losing the ability to prevent the 
unwanted use and rampant dissemination of their work—a consideration that is particularly apt 
where, as here, a defendant places works online for copying and redistribution by numerous third 
parties. (Plfs. Mtn. at 53-54 [Dkt. 60-1].) Public Resource did not rebut this evidence. 

Despite Public Resource’s uncorroborated statements to the contrary, Plaintiffs continue 
to actively sell the 1999 Standards. (Plfs. SDF at ]fl[ 40-41; Levine Deck, ][ 20, Exh. QQQ). 
Public Resource’s allegation that Plaintiffs do not seek any business opportunities with respect to 
the 1999 Standards is wholly unsupported (See Dft. Mtn. at 56). Sales revenue from prior 
versions of Plaintiffs’ standards are vital to Plaintiffs’ financing of future updates (Plfs. SDF at 
51, 66). A loss of revenue from selling the 1999 Standards would result in a lack of funding 
for future revisions of the 2014 Standards and beyond (Plfs. SDF at ^[ 51). 

The consideration of widespread future infringement is particularly pressing in situations, 
like this one, involving digital distribution of a plaintiffs work that can start a potential chain- 
reaction of infringement. “When digital works are distributed via the Internet, every downloader 
who receives one of the copyrighted works is in turn capable of also [re]transmitting perfect 
copies of the work[]. Accordingly, the process is potentially exponential rather than linear, 
threatening virtually unstoppable infringement of the copyright.” Metro-Goldwyn-Mayer Studios, 

-44- 
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Report of the Advisory 
Commission on 
Accessible Instructional 
Materials in 
Postsecondary 
Education for Students 
with Disabilities 


December 6, 2011 
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I know the mandate for the 
Commission was to look principally 
at print material, but the definition 
of textbook has changed. If you 
don’t look at multimedia, you will 
be doing all of us a terrible 
disservice. 

Postsecondary ADA Coordinator 
(2011, July 12) 


The Benefits and Challenges 
of Technology 


barriers to accessing instructional material in 
nonspecialized formats, including an individual 
described in section 121(d)(2) of title 17, United States 
Code [i.e., the Chafee Amendment]. 26 

Another copyright exception that is relevant to the AIM 
discussion is Section 107, commonly known as “Fair Use.” 27 
This doctrine is explained in greater detail in Chapter 1. 

Additionally, the triennial rule-making provisions of section 
1201 of the Copyright Act may be relevant. 28 Section 1201 was 
enacted in 1998 as one part of a copyright amendment known as 
the Digital Millennium Copyright Act (DMCA). 29 It allows the 
Librarian of Congress, upon the recommendation of the 
Register of Copyrights, to exempt certain classes of works from 
the prohibition against circumvention of technological measures 
that control access to copyrighted works, when that 
circumvention is undertaken for certain non-infringing uses 
(e.g., to enable certain e-text controls). 30 This process and some 
of the exemptions of recent rule-makings are summarized in 
Chapter 1 of this report and in more detail in Appendix D. (A 
new rule-making period under section 1201 is currently under 
way; public comments are due December 1, 2011. 31 


The provision of AIM—most commonly in the form of digital 
text, refreshable braille generated from a digital text, embossed 
(paper) braille, tactile graphics, audio, or large print—and of 
access to content in general, is also significantly challenged by 
the emerging importance of digital technologies. In addition, 
online course registration, delivery and assessment; online 
databases, course chat rooms and message systems; open 
educational resources and web pages created by faculty; media- 
rich “textbooks” embedded in popular course management 
systems; computer-based exams used for entrance to or in order 
to complete a course, a major, or a certificate program all 
involve digital technologies. This complex, evolving and 
promise-filled landscape presents an opportunity for 
postsecondary institutions to implement educational practices 
that meet the needs of students who aspire to higher learning 
and improve access for students with disabilities. However, the 
presence of inaccessible technology-based products and services 
within the postsecondary environment can create unintended 
and nearly impenetrable barriers, while the availability of 
products and services that can be accessed by all students, 
including those with disabilities, can open new doors. 

As technology continues to change the instructional materials 
landscape and increases the variety of available course 
materials, digital media has become more commonplace. The 


AIM Commission Report 21 
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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


) 

AMERICAN EDUCATIONAL ) 

RESEARCH ASSOCIATION, INC. et al., ) 

) 

Plaintiffs, ) 

) 

v. ) Case No. 14-cv-857 (TSC) (DAR) 

) 

PUBLIC.RESOURCE.ORG, INC., ) 

) 

Defendant. ) 

_ ) 

ORDER 

On Jan. 21, 2016, Defendant Public Resource moved to strike the declaration of 
Plaintiffs expert Dr. Kurt Geisinger, Ph.D. (ECF No. 60-88). The Geisinger Declaration is 
offered in support of Plaintiffs’ economic arguments regarding the harm to their revenue and 
incentives if the court were to find that incorporation of their standards by reference into federal 
regulations revokes or destroys their copyrights, or Defendant was otherwise allowed to continue 
posting the standards on its website. For the reasons stated herein, Defendant’s motion is 
DENIED. 

A district court has ‘“broad discretion in determining whether to admit or exclude expert 
testimony.’” United States ex rel. Miller v. Bill Harbert Int’l Constr., Inc., 608 F.3d 871, 895 
(D.C. Cir. 2010) (quoting United States v. Gatling, 96 F.3d 1511, 1523 (D.C. Cir. 1996)). Under 
the Supreme Court’s decision in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 
(1993), this court is “required to address two questions, first whether the expert’s testimony is 
based on ‘scientific knowledge,’ and second, whether the testimony ‘will assist the trier of fact to 
understand or determine a fact in issue.’” Meister v. Med. Eng’g Corp., 267 F.3d 1123, 1126 
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(D.C. Cir. 2001) (quoting Daubert, 509 U.S. at 592). Trial courts “act as gatekeepers who may 
only admit expert testimony if it is both relevant and reliable,” Heller v. D.C., 952 F. Supp. 2d 
133, 139 (D.D.C. 2013), though this role is “significantly diminished” at the summary judgment 
stage, see Window Specialists, Inc. v. Forney Enters., Inc., 47 F. Supp. 3d 53, 60 (D.D.C. 2014). 

In determining whether to strike an expert report, the court’s focus is therefore whether 
the expert’s assumptions “amount to ‘rampant speculation’ and should be excluded” or “merely 
represent a weak factual basis for his testimony” which could be appropriately challenged on 
cross examination at trial. Boyar v. Korean Air Lines Co., Ltd., 954 F. Supp. 4, 7 (D.D.C. 1996). 

As the Court in Daubert instructed, “[vigorous cross examination, presentation of contrary 
evidence, and careful instruction on the burden of proof are the traditional and appropriate means 
of attacking shaky but admissible evidence.” 509 U.S. at 596. 

Defendant argues that Dr. Geisinger is not qualified to opine on the economic impact of 
copyright infringement on standards developing organizations. Dr. Geisinger has almost four 
decades of experience as a professor and dean or chair of numerous university departments, has 
served on the boards of numerous testing and accreditation organizations, including Plaintiffs 
AERA and APA, and has served in editorial capacities for numerous testing and educational 
publications. While Defendant points out that Dr. Geisinger is not specifically trained in 
economics, the court finds that his extensive experience studying, working for, and chairing 
organizations similar to Plaintiffs’ qualifies him to opine on how copyright infringement may 
impact Plaintiffs’ revenue and how organizations like Plaintiffs’ may be impacted by changes in 
revenue. 

Defendant additionally argues that Dr. Geisinger failed to sufficiently consider certain 
explanations for the decline in revenue, compared data from different years incorrectly, and 

2 


JA3257 


Case l:14-cv-00857-TSC Document 115 Filed 09/21/16 Page 3 of 3 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 236 of 441 

failed to adequately support his conclusions regarding the impact of potential lost sales on 
Plaintiffs’ revenues and business model. However, Defendant could have retained its own 
rebuttal expert to perform their preferred economic analyses, but chose not to do so. The court 
will not strike an expert report simply because the expert did not rely on the particular 
assumptions or data Defendant thought was necessary. Such alleged deficiencies are more 
properly worked out and probed in “vigorous cross-examination [and] presentation of contrary 
evidence.” Daubert, 509 U.S. at 596. 

Finally, Defendant points to several statements in the Geisinger Declaration that it claims 
were not included in the initial expert report and must be stricken for lack of notice. However, 
the court notes that the contents of the Declaration are simply extensions or reaffirmations of Dr. 
Geisinger’s expert report, and Defendant had an opportunity to probe the opinions and 
conclusions contained in the report at Dr. Geisinger’s deposition. There does not appear to be 
any unfair surprise or lack of notice here. 

The court finds that Plaintiffs have sufficiently established that Dr. Geisinger has the 
experience necessary to be offered as an expert in this case, and the content of his testimony— 
applying his personal knowledge and experience to the effects of copyright infringement of 
Plaintiffs’ 1999 Standards—may “help the trier of fact.” See Fed. R. Evid. 702; Daubert, 509 
U.S. at 588. The court therefore denies Defendant’s motion to strike the Geisinger Declaration. 


Date: September 21,2016 




“S', 


TANYA S. CHUTKAN 
United States District Judge 
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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN SOCIETY FOR 
TESTING AND MATERIALS, 
et al., 

Plaintiffs, 
v. 

PUBLIC.RESOURCE.ORG, INC., 
Defendant. 


AMERICAN EDUCATIONAL 
RESEARCH ASSOCIATION, INC., 
et al., 

Plaintiffs, 
v. 

PUBLIC.RESOURCE.ORG, INC., 
Defendant. 


CA No. 13-1215 (TSC) 


CA No. 14-0857 (TSC) 


Washington, D.C. 

Monday, September 12, 2016 
9:12 a.m. 


TRANSCRIPT OF MOTIONS HEARING 
BEFORE THE HONORABLE TANYA S. CHUTKAN 
UNITED STATES DISTRICT JUDGE 


APPEARANCES : 

For Plaintiff ASTM: J. KEVIN FEE, ESQ. 

JORDANA S. RUBEL, ESQ. 

Morgan, Lewis & Bockius, LLP 
1111 Pennsylvania Avenue, NW 
Washington, D.C. 20004 
(202) 739-3000 

For Plaintiff NFPA: KELLY KLAUS, ESQ. 

ROSE L. EHLER, ESQ. 

Munger, Tolies & Olson, LLP 
560 Mission Street, 27th Floor 
San Francisco, California 94105 
(415) 512-4000 


JA3259 







Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 2 of 142 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 238 of 441 2 


For Plaintiff ASHRAE: 

JOSEPH R. WETZEL, ESQ. 

J. BLAKE CUNNINGHAM, ESQ. 

King & Spalding, LLP 

101 2nd Street, Suite 2300 

San Francisco, California 94105 
(415) 318-1200 

For Plaintiffs AERA, 
APA, NCME: 

JONATHAN HUDIS, ESQ. 

NIKIA L. GRAY, ESQ. 

Quarles & Brady, LLP 

1701 Pennsylvania Avenue, NW 
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Washington, D.C. 20006 
(202) 372-9600 

For Defendant 
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CORYNNE MCSHERRY, ESQ. 

MITCHELL L. STOLTZ, ESQ. 

Electronic Frontier Foundation 

815 Eddy Street 

San Francisco, California 94109 
(415) 436-9333 


ANDREW P. BRIDGES, ESQ. 

MATTHEW B. BECKER, ESQ. 

Fenwick & West, LLP 

555 California Street 

San Francisco, California 94104 
(415) 875-2300 


DAVID E. HALPERIN, ESQ. 

1530 P Street, NW 

1st Floor 

Washington, D.C. 20005 
(202) 905-3434 
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Transcript produced by 
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PROCEEDINGS 

THE DEPUTY CLERK: Your Honor, this is civil case 
13-1215, American Society for Testing and Materials, et al., 
versus Public.Resource.org, Incorporated; and civil case 14-857, 


American Educational 

Research Association, Inc., et al., versus 

Public.Resource.org. 

Incorporated. 

Counsel, please 

come forward and state your appearance for 

the record. 



MR. FEE: Good morning. Your Honor. Kevin Fee on 
behalf of ASTM International. I'm joined at counsel table by 
Jordana Rubel, and we also have general counsel of ASTM in the 
back, Mr. Tom O'Brien. 


THE COURT: 

Good morning. 

MR. KLAUS: 

Good morning. Your Honor. I'm Kelly Klaus 


from Munger, Tolies & Olson representing the National Fire 
Protection Association. I'm joined at counsel table by my 
colleague, Rose Ehler. Our general counsel, Sally Everett, is 


also in the audience 

this morning. 

THE COURT: 

Good morning. 

MR. WETZEL: 

: Good morning, Your Honor. Joe Wetzel 

from King & Spalding 

on behalf of ASHRAE, and I'm joined by 


Blake Cunningham from my firm at counsel table. 

THE COURT: Good morning. And, counsel, I'm going to 
ask you, when you come up to argue, for you to restate your 
name, just because there's so many of you, for my court 
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reporter. 

MR. HUDIS: Good morning. Your Honor. Jonathan Hudis 
for the plaintiffs in the 14-857 case. With me is Nikia Gray, 
and sitting in the audience is immediate past general counsel, 
Nathalie Gilfoyle, and current general counsel, Deanna Ottaviano. 

THE COURT: Good morning. 

MS. MCSHERRY: Good morning. Your Honor. 

Corynne McSherry for Public.Resource.org, and with me at the 
counsel table is Andrew Bridges, my co-counsel who will also be 
arguing part of the case, the bulk of the issues, to be honest. 

He took that all on. Also with me at counsel table is Matt 
Becker of Fenwick & West; Mitch Stoltz, with me from the 
Electronic Frontier Foundation; and David Halperin. 

THE COURT: Good morning, everyone. I know that the 
parties had wanted more time; and I've given them less time than 
they wanted, but I'm confident, having been through the 
materials, that we can accomplish everything we need to 
accomplish today. 

I'm just going to ask that you be mindful of probably one 
of the more important people in this room, which is my court 
reporter, Mr. Wayne, who has to get all this down. So I'm going 
to ask you, again, announce yourselves when you come up to the 
podium and to speak clearly and not too quickly, something I 
have to remind myself of as well. 

MR. HUDIS: Your Honor, in light of the reduced time, 
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we gave your clerk our proposal for scheduled arguments. 

THE COURT: I saw that. That's fine. 

All right. So let's get right on it. I have some 
questions, obviously, but I will raise them when it seems 
appropriate as you're in your argument. So it looks like we're 
going to deal with copyright issues. ASTM is going first. 

Is that you, Mr. Klaus. 

MR. KLAUS: Yes. Kelly Klaus representing NFPA 
and speaking on behalf of all the plaintiffs in the ASTM case 
on the copyright issues other than ownership. Your Honor, and 
being mindful of time. I'll keep my own clock out and try to 
watch. 

THE COURT: All right. 

MR. KLAUS: I told Mr. Hudis that I would try to 
roughly hew to his schedule. I also told him that if I happen 
to go over by a few minutes, we're happy to take time off of 
other things on the back end. 

THE COURT: All right. 

MR. KLAUS: But we'll try to keep it there. 

Your Honor, as you noted, you have a mountain of paper 
that's been presented to you, and we appreciate the Court's 
patience in reviewing all of it notwithstanding the numerous 
materials that are here. 

This is, we think, a very straightforward case of copyright 
infringement. There are numerous works that have certificates 
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of registration that come with the presumption of ownership and 
validity, and we have a defendant who is engaged in wholesale, 
100 percent, verbatim exercise of multiple of the exclusive 
rights of copyright. It has engaged in unrestricted, and until 
he voluntarily stopped, pending resolution of these issues 
before Your Honor, unrestricted distribution of these works. 

I'd like to cover — there are numerous copyright issues 
other than ownership in the case, and numerous cases. I'd like 
to cover three broad areas, and I'm happy. Your Honor, to 
address, of course, all the questions that you have on these. 

But the three areas, first, are that we think this is a 
clear case where Congress — this is a case of congressional 
intent, ultimately, and this is a case where we think Congress 
has spoken and that the Copyright Act makes clear that the works 
in question are copyrighted and do not lose their copyright 
protection simply because they are incorporated by reference. 

The second is, I'd like to address the split in the case 
law which is at the center of the dispute, really, between the 
Veeck case from the Fifth Circuit — 

THE COURT: And I'm sorry. Just for Mr. Wayne's 
purposes, Veeck is spelled V-e-e-c-k, and there's one other 
phrase that's going to probably come up that I had to consult my 
French-speaking husband for, which is scenes a faire, which is 
s-c-e-n-e-s a f-a-i-r-e. So those are just for the transcript. 
Okay. 
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MR. KLAUS: Thank you. Your Honor. And the split 
between the Veeck case on the one hand and the Ninth and Second 
Circuit cases, that subsumes within it a number of issues 
including merger and the idea-expression dichotomy. 

The third issue that I'd like to cover briefly is the fair- 
use defense that's been raised and why we think that that can be 
resolved as a matter of law now. 

Your Honor, we think that ultimately, turning to the first 
point, this is really a question of what did Congress intend in 
the Copyright Act. There are numerous arguments that we have on 
the other side that have a lot of rhetoric behind them in terms 
of there being an impingement of due process rights, an 
impingement of the right of people to speak or to think about 
the law. 

I would note a point that I will come back to several 
times. There is — notwithstanding multiple years of litigation 
with discovery into numerous issues, there is absolutely no 
evidence to back up any of the claims that anyone has ever been 
deprived access to the standards in issue. 

There's no evidence that anyone has been deprived of 
their First Amendment right to speak, much less that those 
constitutional claims could be asserted against the property 
rights of the plaintiffs in this case who have their own 
constitutional issues lurking in the background in the form of a 
potential taking by state action that would appropriate their 
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property and their copyright. 

So you have constitutional avoidance questions that, at 
best for the other side, cut in both directions, and really I 
think ultimately counsel the Court back to the plain words and 
the plain intent of the copyright statute. 

Now, under § 102(a), it's plain that when the 
underlying works are created, they meet all the standards for 
copyrightability. They are original. They meet the original 
requirements that were laid out by the Supreme Court in the 
Feist case. They certainly have much more than a minimal degree 
of creativity to them. 

THE COURT: Mr. Klaus, does it matter if the standards 
were created with anticipation or with the expectation that they 
would be incorporated into law? 

MR. KLAUS: No. 

THE COURT: And why not? 

MR. KLAUS: Because, first of all, what the evidence 
actually shows is that that is — and it's undisputed on behalf 
of all the plaintiffs that the standards are used for multiple 
purposes other than simply being created — simply being enacted 
into law. And they in fact have numerous uses that they are 
used by people outside of simply a matter of legal compliance, 
and those points are set forth in Mr. Thomas's declaration, 

Mr. Pauley's declaration, and Mr. O'Brien's declaration. 

So these are not standards that are solely, or as in the 
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hypothetical case that the defendant has raised, for example, a 
K Street lobbyist who takes something to his or her favorite 
legislator for no purpose other than to have that item enacted 
into law. There's not a tradition of copyright protection for 
such materials as there is copyright protection for these 
materials. And the policy considerations, we'd say, are 
completely different. 

So we think the standards here were original in that they 
have the minimum amount of creativity, they were not copied from 
anywhere else, and there's nothing in the statute that says they 
are not copyrightable when they are created. 

To the extent that the statute speaks at all about 
copyright protection for works that overlap with law, it's in 
§ 105 which says that works of the United States government, 
meaning a work that's created or prepared by an officer or 
employee of the United States in the course or scope of that 
person's duties, are not subject to copyright protection. 
Otherwise, nothing in the statute says that incorporation by 
reference divests the standards of protection. 

And we know from other legislation. Your Honor, specifically 
the National Technology Transfer Advancement Act, NTTAA for short, 
of 1995, specifically in 15 U.S.C. § 272(b)(3), specifically 
expresses a preference for standards for federal agencies to 
incorporate by reference. 

There are a variety of policy reasons that underlie that. 


JA3267 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 10 of 142 
Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 246 of 441 10 


but there is a recognition in that statute and in the continued 
decisions of federal agencies, including the Office of the 
Federal Register, including the Office of Management and Budget, 
in our request for Judicial Notice No. 1, which is the Circular 
A-119, that express a clear preference for federal agencies to 
rely on voluntary consensus standards, numerous policy reasons 
underlying that, numerous policy reasons that we think frankly 
undercut a number of the parade of horribles of the lack of 
transparency or accountability in government decision-making. 

For example, the fact that voluntary consensus standards are 
open to the public. There's not a danger of industry capture of 
voluntary consensus standards. 

And repeatedly. Public.Resource has made the same arguments 
that it's making to this Court about the fact that incorporation 
by reference necessarily divests the copyright to OMB, to the 
Office of the Federal Register, and those arguments have been 
repeatedly rejected. 

Now, I'd like to turn, if I could, to the heart of the case 
law dispute. 

THE COURT: Mr. Klaus, let me ask you or your clients, 
are they currently for sale, the standards at issue in this 
case? You currently sell the standards? 

MR. KLAUS. The standards as we publish them? 

Correct. We do. 

THE COURT: Is there any objection to sell the 
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standards since they're incorporated by reference? In other 
words, do you have to sell them? 

MR. KLAUS: That's an interesting question as to 
whether they would have to be sold. If this were a case where 
some governmental body incorporated the standard by reference, 
and if the standard-setting organization said we're not going to 
sell them, that would be a — not only would it be a very 
different case; you would probably come closer, at least at a 
minimum, on the fair-use case to something like, for example, 
the Swatch case that the defendants cite, and that's the case 
where the company didn't want — they claimed copyright 
protection over the transcript of its earnings recording. But 
it did that for the purpose of — 

THE COURT: It's a close call. 

MR. KLAUS: — keeping it out of the public record. 

And so what the interest of the copyright owner in that case was 
trying to preserve had nothing to do with what the purposes of 
copyright are. 

THE COURT: But if you stopped selling the standards, 
is it still reasonably available under the OFR's regulation, 
especially if the regulation incorporating the standard by 
reference says that it's available from the authorizing 
organization? 

MR. KLAUS: I think it would be a very hard case for 
me or for anyone else to make if the standards weren't 
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available. 

THE COURT: Okay. I have another question, but I'm 
going to wait till you get to that. 

MR. KLAUS: Sure. Let's talk briefly about the 
distinction between the Veeck case and Practice Management and 
the CCC case from the Second Circuit. Ultimately, that is the 
main argument that Public.Resource advances here, which is that 
this Court should follow the majority opinion of the en banc 
Fifth Circuit in the Veeck case. 

And I think it's important to emphasize at the outset of 
this. Your Honor, there really are two lines of cases here. 

There are two lines of cases that deal with the same issue. 

There is a split of authority, and ultimately the Court has 
to decide which one is the more persuasive of the two. 

It's our position that the better reasoned cases, the cases 
that are more sensitive to the precedent and to the policy 
considerations here are Practice Management and CCC . With 
respect to j|ia-c||5id€f iteffagem^fft , that's the Ninth Circuit case 
that involved the HCFA regulations that incorporated by 
reference the AMA's CPT. My apologies for all the acronyms 
here. 

In the Practice Management case. Your Honor — first of 
all, let's be very clear. Practice Management is not, as we 
see some reference to it in the defendant's briefing and as 
there were some references to it in the Veeck case trying to 
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distinguish it, a case about simply referring to some numbers 
that the ABA published. There was a system that — an entire 
coding system that the AMA had, and the coding system — 

THE COURT: The AMA. 

MR. KLAUS: AMA. 

THE COURT: Okay. I thought you said ABA. 

MR. KLAUS: My apologies if I did. There are a lot of 
letters to keep up with. 

THE COURT: The ABA is not organized. 

(Laughter) 

MR. KLAUS: We may get a lot of stipulation for that. 
Your Honor. 


THE COURT: All right. 

MR. KLAUS: The AMA's standards were incorporated. 

The Ninth Circuit said so explicitly in the opinion. There were 
federal regulations that incorporated those by reference. 

Someone who wanted to be reimbursed for expenditures that were 
reimbursable under Medicare, Medicaid, had to use that system. 
There is no difference between that and the types of standards 
that are at issue here. 

And as the Ninth Circuit said in that case, ultimately the 
question, they said, boiled down to whether or not the Banks 
case from the 1800s established some divestiture of copyright. 
This is a major point of difference between the Ninth Circuit 
and the Fifth Circuit. What the Ninth Circuit recognized about 
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Banks , we think correctly, is that Banks is a case that says, 
for purposes of the copyright statute, judges aren't authors. 
Judges, in the course and scope of the opinions that they 
write — we certainly know judges can and do create things 
outside of what they do and get copyright, but in the course and 
scope of writing opinions, it's not subject to copyright 
protection. 

THE COURT: No matter how celestial the prose. 

But let me ask you, didn't the Ninth Circuit, when they 
looked at Banks , it focused on Banks ' premise that there's a due 
process premise in fair access to law. It seemed that the Court 
in the Ninth Circuit considered the due process interest and 
rejected it because of the fact that there was no evidence that 
anyone wishing to use the copyrighted codes had any difficulty 
obtaining access to it. 

MR. KLAUS: Correct. 

THE COURT: Is that what you're arguing here? 

MR. KLAUS: That's the second ground that they 
discussed, and that's also — that's a point of departure with 
the Veeck case. The Veeck majority said, we don't want to look 
at evidence of availability or accessibility. Don't put that in 
front of us; we don't care. We read Banks as establishing a 
continuous tradition which we would submit, respectfully, there 
is no continuous tradition of standards incorporated by 
reference not being protectable. 
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But Practice Management does indeed say that accessibility, 
that there is a due process consideration, and there's a 
question that if somebody has to comply with a legal requirement, 
can they have access to it. And this is very critical here, 

Your Honor. There is no evidence — again, after years of 
discovery and litigation, there is no evidence that anyone 
who has needed to comply with any of the standards that the 
plaintiffs in this case publish or that are at issue here, 
there's no evidence here that anyone has ever had any problem 
gaining accessibility to any one of those standards. 

In fact, the standards are all made freely available online 
in online reading rooms. So, if anyone wanted to know what is 
the particular requirement, they can go to the Internet and all 
the standards are completely available. Again, not a shred of 
evidence on the other side that there has been any problem of 
accessibility. 

THE COURT: How do the standards here differ from the 
model codes that were at issue in Veeck ? 

MR. KLAUS: Well, the standards that are at issue 
here were not — and this goes to a question. So, first of all, 
with respect to Veeck , the language that's in the majority 
opinion — and it's important to focus this. This is on page 
293 F.3d at page 805. What the majority said here is that the 
standards — in the case of a model code, reading in the first 
column, model code which the majority says is not protected by 
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copyright, loses its copyright protection when it's incorporated. 
The text of the model serves no other purpose than to become 
law. The characterization that the Court put on in the way it 
decided the case was to say that SBCCI, the acronym for the 
plaintiff there, operates with the sole motive and purpose of 
creating codes that will become obligatory law. 

And in fact, at the end, what the Court says — and the 
Court says the result in this case would be different but 
recognizes we're potentially creating a circuit split and this 
is the way out, is to say that we will characterize these codes 
as having no purpose other than having been enacted to become 
positive law. 

And here, Your Honor, the undisputed evidence is that 
that's not the sole purpose that the plaintiffs enact the codes 
for. The evidence is that they are in fact used by business and 
industry for purposes other than simply law, and there's not the 
sole expectation that they will simply become law and simply be 
incorporated and wholesale adopted. 

In fact, what the evidence actually shows — and this is 
discussed at length in the ICC^ c. ~ International Code Council 
amicus brief, is that actually numerous of the standards, 
including the standards at issue here, when they are 
incorporated by reference, federal agencies, state agencies may 
adopt a portion of them. 

For example, in th^.PractjQ.d ‘ Mafagi.gemd.hh T - case itself, you'll 
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see there's a reference to my client's standard, the National 
Electrical Code, and there's a citation to a particular federal 
regulation that doesn't incorporate the entire thing by reference 
but incorporates particular portions of it. 

There are other jurisdictions that may incorporate and make 
various changes and amendments to them, but it's not the 
paradigm that you have referenced in the Veeck case, which is 
something that is simply put forward solely for no other purpose 
than to become law. 

The other point of distinction between Practice Management 
and CCC and the Veeck case that we think is important is Veeck 
starts out by saying, here's a Supreme Court opinion in Banks . 
We're not going to look at it just as a matter of statutory 
construction. We're going to say this settled the matter once 
and for all in the 1800s, that anything that's incorporated by 
reference automatically becomes the uncopyrightable law, free to 
all to use. 

But there was a backup argument that the majority put in 
which was, even if it doesn't, there's a merger. At the moment 
that a standard is incorporated by reference, the fact merges 
with whatever is capable of being the copyrightable expression. 
And we see the merger argument raised by the defendant in this 
case, and the merger argument, we think, was quite properly 
rejected in Practice Management . As the court said, the point 
of merger is that, at the moment of creation, what was the 
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constraint on the author? 

THE COURT: Your argument is that the merger — for 
the merger doctrine to apply, the merger doctrine analysis takes 
place at the instant of the work's creation. 

MR. KLAUS: That's correct. And that's — the most 
recent exposition of this was in the federal circuit's decision 
in the Oracle v. Google case which deals with computer software 
which has its own, in some ways, sui generis copyright analysis. 
But the important point there is the merger discussion isn't 
limited there, and it's also — you see it in the Practice 
Management case. 

The question of merger is, we don't want the very first 
person who writes "roses are red, violets are blue" to have 
a copyright on the saying that roses are red. That is simply 
taking that idea out there and removing it from circulation 
because there are a minimal number of ways that any author could 
have expressed that expression. 

THE COURT: So there has to be no other ways of 
articulating a particular idea when the work is first published. 

MR. KLAUS: That's correct. And we know in this 
case — the record is clear in our case that, in fact, there are 
other organizations who create standards on the same topics 
here. I'd refer Your Honor to our statement of undisputed facts 
38 and 133 by way of example on that. There's no dispute that 
there is no constraint on any of the organizations here in terms 
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of their authorship, in terms of the types of creative, 
expressive choices that they would have to make at the moment 
of creation. 

Practice Management , at 121 F.3d at page 520 in footnote 8, 
specifically says this is the reason why we are not going to 
apply the merger doctrine here. Judge Leval's opinion for the 
Second Circuit in the CCC case is to the same effect, 44 F.3d at 
72. What he says is merger is a judicially created doctrine, 
and we will decide how and when to apply it depending on what 
are the needs to leave the breathing room for creativity and 
expression. 

Just in the last few minutes that I'm going to try to take 
for my time. Your Honor, let me talk about fair use, and the 
main points I'd like to make on fair use are that the use here, 
however it's described, is — and whatever the purposes that are 
claimed, is plainly substitutional. This is a defendant who is 
engaged in the business of making wholesale copies, distributing 
those — 

THE COURT: What aspects of the defendant's actions 
are commercial as opposed to political? 

MR. KLAUS: Well, the question that the Supreme Court 
tells us in Harporp & Row- , is that the distinction between 
commercial/noncommercial is not whether somebody says I'm out 
to make a huge profit. It's not whether I'm General Motors or 
whether I'm the NRDC. The distinction is whether you are 


JA3277 





1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 20 of 142 
Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 256 of 441 20 


exercising a right that customarily one would have to pay for in 
that context. But regardless of whether he's commercial or 
noncommercial, the question really on the first fair-use factor. 
Your Honor, on the transformativeness test — 

THE COURT: I have a question. What would be a 
transformative use of your standards? 

MR. KLAUS: Well, I could — I could certainly imagine 
somebody writing, for example, an article about critiquing the 
standards. I could certainly imagine somebody writing an 
academic piece that would say I've got a -- I've got a problem 
with this or here's how the standards have developed in this 
area. I could certainly imagine numerous fair uses. 

And that's one of the important points, Your Honor, is on 
fair use, the answer to the parade of horribles that we have 
from the other side about people being thrown in jail for 
speaking the law, about people being subject to massive statutory 
damages awards for daring to write. 

The idea that copyright is somehow this omnipresent force, 
that once it's conferred there exists this pressure that will 
inevitably lead to people stopping talking about whatever the 
standards are that have been adopted by jurisdictions, that's 
just not true, and there would be plenty of cases of fair use 
that would be perfectly fine. 

There are plenty of uses that people can make of the works 
in question. The issue is that the defendant's work here, what 
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the defendant is doing is entirely substitutional. They have 
made the entire works available for copying for distribution 
without limitation. 

With all due respect to my friends on the other side, there 
is no case in the history of fair use that has come close to 
saying that a defendant who creates — who engages in that sort 
of verbatim copying and makes the entire work available in a 
manner for copying for downloading, for distribution, that that 
is in any sense for fair use. 

And the two cases I would direct you to. Your Honor, the 
most recent cases on this are — calling them the Authors Guild 
case does not help, because they're both Authors Guild cases 
from the Third Circuit. But one is the HathiTrust , and one is 
the Google Books case. 

In the HathiTrust case, one of the claims of transformation 
was that the HathiTrust had made searching easier for works. 

It had a transformative purpose or function because the copying 
made the works more easily searchable. That's one of the 
arguments, by the way, that the defendant has raised. He said, 
well, I, by converting these to HTML — 

THE COURT: They're visually more searchable. 

MR. KLAUS: Right, which I should also add 
parenthetically, the evidence in the record is that a number 
of the standards here are also made available in HTML and XML. 
That is part of a license that one has to look to. I would 
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refer Your Honor to Mr. Thomas's declaration at paragraph 44 in 
that regard. 

The important point in the HathiTrust case, though, is the 
court went out of its way to say no copy of the work is made 

available as a result of the searching. So the transformation 

that was done to enable searching allowed the computer, behind 
the scenes, to find something and to refer the user to the 
particular work, but it didn't make an exact copy available. 

The Google Books case, also an Authors Guild case, also 
from the Second Circuit, there's a wrinkle in the Google Books 
case, which is that Google not only made searching easier by its 
copying, but it also provided snippet view, which is — 

THE COURT: And that was what was found to have 

transformative — was found to add value to the transformative 

search function. 

MR. KLAUS: It was found to add value to the 
transformative search function, but Judge Leval went out of his 
way to say that the snippet view did not operate as a substitute 
for the work. There were a number of precautions that Google 
had put in place that it would — for example, when a word 
search was done, it would return only the same portion of the 
work. One couldn't game the system by putting together multiple 
snippets and get the work. Works that were very short were 
excluded from the snippet view so that somebody couldn't game 
the system that way. Authors who wanted their works out could 
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opt out of the system. 

And Judge Leval's quite clear that it was putting this 
mechanism in that made the difference, and he in fact 
specifically said that at 804 F.3d 217, that if the function — 
not the purpose. Because the purpose, Public.Resource says, 
well, we're making all of your works available, but we're doing 
it for a different purpose because we just want the law out 
there, and therefore we win on the first factor. 

And what the Second Circuit said is, no, when you're 
engaged in verbatim copying, the question as to whether or not 
you win on that first factor is not what your purpose is, not 
what your intent is; it's whether the function of what you're 
doing is exactly the same as what the copyright owner does. 

We'd say you have exactly the same thing here. 

That conclusion, we think, drives the third factor, 100 
percent copying, 100 percent of the work made available; and 
also the fourth factor on market substitution, on the fact that 
if this is fair use, if what the defendant here is doing is fair 
use, there is no limitation to anyone doing the same thing. 

One brief other point on fair use, Your Honor. There was a 
claim that was made post hoc that this system was set up to make 
these works available for the visually impaired. Like most post 
hoc justifications, when you actually look at the facts and the 
reality, that wasn't the purpose. What the HathiTrust case 
again points the way here on, there's no question that making 
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works available to the print disabled is an important function. 

The defendant's work doesn't just make the works available 
to the print disabled; it makes it available to the entire 
world. And, again, what the undisputed evidence shows is that 
the one print-disabled person who told one of the plaintiffs — 
my client, in fact — that they had difficulty reading online, 
they were given an entire copy. 

The other plaintiffs have said, if somebody said that they 
had a problem, we would give them a copy, or they could even go 
to what claims to be the Chafee Amendment compliant site that's 
operated by Mr. Fruchterman, who was the defendants' expert, and 
obtain a copy. So we think, for that reason, the fair-use 
defense is completely without merit. 

I believe I've gone over. I'm happy to answer any questions. 

THE COURT: No. I've been peppering you with them as 
we go along. Thank you. 

MR. KLAUS: Thank you very much. 

THE COURT: Mr. Hudis. 

MR. HUDIS: Good morning. Your Honor. Jonathan Hudis 
for plaintiffs in the 14-857 case, American Educational Research 
Association, AERA; American Psychological Association, APA; and 
National Council on Measurement in Education, NCME. 

Your Honor, in our briefs we refer to AREA, APA, and NCME 
as the sponsoring organizations of the 1999 Standards for 
Educational and Psychological Testing, the work that was 
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infringed in this case. 

This is an even simpler case than the ASTM case, Your Honor. 
Public.Resource, operated essentially by one person, Carl 
Malamud, admits he digitally copied plaintiffs' standards and 
published to the Internet for others to download, print, and 
copy for free. 

Public.Resource asks this Court to excuse its acts of 
copyright infringement and contributory infringement as fair 
use, stretching the limits of this defense well beyond its 
breaking point, all while trampling on the copyrights of three 
nonprofit organizations guaranteed to them by the Constitution 
and the Copyright Act, and those are the rights to reproduce the 
work, to prepare derivative works from it, to distribute copies, 
and to display the work publicly. 

It should be noted that, in addition to the copies of the 
standards which can be purchased from the plaintiffs, their 
standards are available at the U.S. Department of Education, the 
Office of the Federal Register, and thousands of libraries 
throughout the country. 

THE COURT: Mr. Hudis, how much does it matter if you 
can get the standards for free already, either through the OFR 
or through libraries or read-only rooms, as you all have? 

MR. HUDIS: Well, Your Honor, I was anticipating your 
accessibility questions as to the ATSM plaintiffs, so we just 
want to put that to rest. 
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THE COURT: Okay. 

MR. HUDIS: So, as a legal matter, the answer is 
nothing if defendant's theory of the case is correct. 

Your Honor, that privately created standards lose their 
copyright upon being incorporated by reference into the 
regulations of an agency. 

As plaintiffs' counsel said in the ASTM case, this Court 
would be sanctioning a widespread taking of copyrighted property 
without just compensation, in violation of the Fifth Amendment. 
The standards development organizations do not have continuing 
financial incentives to promulgate and update their valuable 
works. Important stores of knowledge will no longer be 
available to the public. 

How to resolve the competing interest raised in this 
litigation should be a decision for Congress to make, not the 
court legislating from the bench. In the meantime, this Court 
should uphold the sponsoring organization's copyright and enjoin 
Public.Resource from further acts of infringement. 

THE COURT: Mr. Hudis, I see that the 1999 standards 
weren't sold for a period of time. 

MR. HUDIS: Yes, Your Honor. 

THE COURT: Is there any obligation to sell them since 
they're incorporate by reference into law? 

MR. HUDIS: Your Honor, that does not have a bearing 
on the case, to answer your question. The fact is they are on 
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sale for a period of time so that the 2014 standards could get 
into circulation. The 1999 standards were taken off sale. They 
are now sold again on AERA's website. 

THE COURT: Even during the period in which they were 
not for sale, were they available through OFR or through some 
other means? 

MR. HUDIS: They were available in three places: the 
U.S. Department of Education, through the Office of Federal 
Register, and thousands of libraries throughout the country. 

THE COURT: Okay. 

MR. HUDIS: So the answer's yes. 

Now, Your Honor, plaintiff's work, the '99 standards 
infringed in this case, were a set of best practices of 
guidelines in the creation, administration, scoring and use of 
standardized tests, covering issues such as test validity, 
reliability, comparability, fairness, and other items. The 
sponsoring organizations don't keep the profits from these 
sales, and they use the profits to fund further — 

THE COURT: Does that matter? 

MR. HUDIS: No, it does not. Your Honor. But we 
are entitled to the fruits of our copyrighted work. 

THE COURT: Right. 

MR. HUDIS: Now as to authorship. The '99 standards 
were born from an extensive revision of the 1985 standards by a 
sixteen-member expert volunteer committee. Their work resulted 
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in over 50 percent of new content in the '99 version. 

Although — now, and this is important because it was 
raised in Public.Resource's briefing. Although the drafts of 
the '99 standards were published for comment, and many comments 
to these drafts were received by joint committee, the ultimate 
content of the 1999 standards came from the authorship of the 
joint committee members. 

Public.Resource has not submitted any admissible evidence 
to the contrary, and in fact concedes in its summary judgment 
brief at page 27 that the joint committee controls the final 
product through the text. 

THE COURT: So it's not creation by crowd sourcing or 
anything like that. 

MR. HUDIS: No, it is not. We have unrebutted 
evidence in our record that says that the joint committee was 
the ones who promulgated the final text. They did receive many 
comments, but there is no evidence in the record that those 
comments were incorporated word for word into our standards. 

The final selection of that language was chosen by the joint 
committee members. 

Now, Your Honor, I'll skip over ownership because we 
have that in another segment. Public.Resource confirmed its 
infringing activities in its interrogatories and Mr. Malamud's 
deposition testimony without permission. He bought a used copy 
of the 1999 standards, which I have here. 
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He cut apart its bindings, scanned the entire book to an 
Acrobat Reader PDF file with a self-made certificate, which we 
handed up to Your Honor, and appended the certificate to the 
front, published the PDF file to its own website, and also 
published that file to the Internet Archive site. Importantly, 
Your Honor, and which came up in the other argument, neither 
side precluded users from freely downloading or printing the PDF 
file. These facts are uncontested. 

As to contributory copyright infringement, the self-made 
certificates that you have before Your Honor are appended to 
the front of the unauthorized PDF copy of the standards, 
unmistakably states that the work was incorporated by reference 
into regulations. In Public.Resource and Mr. Malamud's view — 
THE COURT: You said this is a self-created 


certificate? 

MR. HUDIS: Yes. Mr. Malamud created it. 

THE COURT: So this approved seal — 

MR. HUDIS: That's all Mr. Malamud's creation. 

THE COURT: Okay. 

MR. HUDIS: In Mr. Malamud's and Public.Resource's 
view, once incorporated by reference, the standards lose their 
copyrighted protection and thereafter can be freely copied by 
anybody. Therefore, the purpose of the certificate was to give 
the public a false sense of approval or permission to download, 
print, or copy the standards without authorization. 


JA3287 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 30 of 142 
Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 266 of 441 30 


Additionally, and if you would look — 

THE COURT; Well, isn't that one reading of it? One 
interpretation of the self-created certificate is to create an 
imprimtur of officialdom. I mean, it has a seal. It has an 
official incorporator. It has a lot of citations to the Federal 
Register. Isn't one interpretation of a certificate is just to 
confer an imprimtur that it's an approved, official document? 

MR. HUDIS: Until we took the deposition of 
Mr. Malamud, one would agree with you. Your Honor. But the 
purpose of putting up that up certificate was, in his view, 
to tell the public that, upon incorporation by reference, the 
standards were now losing their copyright and freely available 
for everybody. 

And he went even further. Your Honor. When he published 
the standards to the Internet Archive site -- I've put several 
of these in front of you. Your Honor. You can see at the very 
bottom right, before the red at the very bottom, it says 
"Creative Commons License." So we asked him about that at his 
deposition, and he said, "This language included the creative 
commons license, indicating that no rights were being asserted 
over the item." 

So, according to Public.Resource's interrogatory answers 
and discovery taken of the Internet Archive, during the nearly 
two-year period that the PDF file was published to the two 
websites, the standards were accessed several thousand times. 


JA3288 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 31 of 142 
Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 267 of 441 31 


We do not know who accessed the unauthorized, online copies 
because Public.Resource refused to provide its web server logs, 
and our discovery motion seeking their production was denied. 

During the same two-year period that the unauthorized PDF 
file was published online, the sponsoring organizations 
experienced a precipitous drop in the sales of their standards, 
which is inconsistent with a work of this longevity where we 
typically would have seen a gradual year-over-year sales decline, 
according to our expert. Dr. Geisinger. While ongoing work on a 
new edition of the standards, ultimately published in 2014, was 
announced during this period, this does not explain away the 
considerable sales drop. 

While members of the sponsoring organizations might have 
wanted to wait for new editions of the standards, psychometrics 
and educational testing students could not wait because the 1999 
standards were still being assigned as cross-reading material. 

We had become aware that students were obtaining free copies 
online with Public.Resource as their source. 

Now as to harm. Public.Resource still has the 
unauthorized PDF file of the '99 standards in its possession. 

If Public.Resource is successful in this suit, defendant can 
easily republish the file to the Internet. 

Further, Public.Resource's every intention, if allowed by 
this Court to do so, is publishing the sponsoring organizations' 
2014 standards to the Internet once incorporated by reference 
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into government regulations. 

THE COURT; That's where your irreparable injury and 
continuing harm argument comes in. 

MR. HUDIS: Yes, Your Honor. And future harm to the 
sponsoring organizations includes loss of future income to fund 
further revised editions of the standards and public confusion 
that the '99 standards are the current version of the standards 
published by the sponsoring organizations, when they're not. 

High-stakes tests, Your Honor, the gateways to educational 
matriculation and attaining employment, must be properly 
designed, administered, scored, and relied upon. There is thus 
a high-societal value for the continuing update of the standards, 
an important body work, produced for the general public. 

Now, what could have Public.Resource done differently? 
Public.Resource makes a red herring argument, as it did in the 
ASTM case, that the purpose of its infringing activity was to 
make the sponsoring organization's standards available to the 
blind or people with less severe print disabilities. 

If that was truly the case. Your Honor, Public.Resource 
could have narrowly tailored access to plaintiffs' standards 
to individuals with certified blindness or print disabilities 
as provided under the Chafee Amendment such as Braille, 
audiotape/CD availability, large font, video, screen, or closed- 
circuit TV magnification, color contrast choices, human readers, 
or limited search term availability as we discussed in the 
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HathiTrust and the Google cases. 

Public.Resource could have imposed limitations on 
the availability by methods as access — methods of access by 
credentials such as a user name, password, digital rights 
management, fingerprint tracing of unauthorized downloads, and 
access terms and conditions, all which was testified to by 
defendant's expert, Mr. Fruchterman, that he and his company 
practice on the Bookshare-Benetech website. 

Your Honor, so we have met our elements of the cause of 
action. We have a valid copyright, which is conceded, 
copyrights of the entire work. We complied with the statutory 
formalities of registration. We also here have copying as a 
factual matter, and copying of the copyrighted material was so 
expensive that it rendered the offending and copyrighted works 
identical. 

THE COURT: Mr. Hudis, what evidence do you have of 
direct third-party copyright infringement? 

MR. HUDIS: Your Honor, a good question. 

Now direct copyright by third parties. Thousands of 
Internet users access the standards on Public.Resource's 
Internet Archive's website. We have that from the deposition 
testimony and the interrogatory answers of Public.Resource, and 
we have the deposition testimony of Internet Archive. 

THE COURT: But that's based on like hits to the 
website; right? What's your direct evidence that people 
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actually downloaded this material? 

MR. HUDIS: Okay. So, Your Honor, the one piece of 
evidence that we were looking for for that were the web server 


logs. We never got 

them. 

THE COURT: 

: Right. You submitted, I think, an 

affidavit — or not 

an affidavit but an e-mail. 

MR. HUDIS: 

: Yes. Two sets of e-mails. 

THE COURT: 

: From, I think, a professor saying that the 

students got it off 

the — 

MR. HUDIS: 

: Yes. 

THE COURT: 

: But notwithstanding admissibility 


questions, because — 


MR. HUDIS: 

: That's hearsay. Your Honor. 

THE COURT: 

: Well, you're right. Is that it? 

MR. HUDIS: 

: No. No. So the users who pulled up 


the standards on their web browsers displayed the copyrighted 
material, at which time the copies were made in the random 
access memory of their computers to permit viewing of the 
materials. By displaying the work and making those copies, 
even temporary ones, those users directly infringed the 
copyright. Your Honor, during that same period of time is when 
we experienced the precipitous drop in our sales. 

So, Your Honor, we took as much circumstantial evidence 
as we could give to you to muster. We have the period of time 
from mid-2012 to mid-2014 when the standards were up on the two 


JA3292 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 35 of 142 
Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 271 of 441 35 

websites. We have the proof of access, and we have the proof 
that the sales went down. That's our circumstantial case 
because we never got the web server logs. And I am sure learned 
counsel for the defendant will tell us that we're all wet on 
that, but that is our circumstantial evidence. 

Your Honor, Public.Resource contends that a copy for 
purposes of copyright is limited to physical objects and thus 


did not make a copy 

of the standards in the legal sense. That 

is absolutely false. 

The infringing version stored on 

Public.Resource and 

Internet Archive's web servers are copies 

for the purposes of 

the Copyright Act. Electronic copies of the 


work stays on computer. Computers, with their RAM memories, are 
copies under § 101 of the Copyright Act. 

I've gone through the evidence of reproducing, of creating 
derivative works, of distribution. Your Honor, I would like to 
now turn to Public.Resource's defenses, unless you have any 
questions. 


THE COURT: 

: That's fine. 

MR. HUDIS: 

: All right. Your Honor, Public.Resource 


does not need to access the standards, free or paid for, in 
order to comply with any of the government regulations or laws 
Public.Resource claims it has the right to post copies — 


THE COURT: 

: You have to slow down again, Mr. Hudis. 

MR. HUDIS: 

: Slow down? 

THE COURT: 

: A little bit. 
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MR. HUDIS: All right — of our standards online so 
that others can copy, print, distribute, or otherwise use them 
for free. All of the cases relied upon by Public.Resource are 
distinguishable. Wheaton v. Peters , Supreme Court decisions. 
Banks v. Manchester , Ohio Supreme Court decision; Howell v. 
Miller , Michigan state statutes; and let's talk about the Veeck 
case which you brought up with my learned co-counsel. 

Veeck involved a word-for-word reproduction of model 
building code into legislation which does not apply to the 
incorporation by reference of extrinsic standards, making Veeck 
inapplicable in reasoning and result. The holding of Veeck is 
that the law, whether articulated in judicial opinions or 
legislative acts or ordinances, is in the public domain. 

Importantly, Your Honor, at pages 803 and 804, Veeck says 
clearly, "The limits of this holding must be explained. Several 
national standards-writing organizations fear that copyrights 
may be vitiated simply by the common practice of governmental 
entities' incorporating their standards in laws and regulations. 
This case does not involve references to extrinsic standards. 
Instead, it concerns the wholesale adoption of a model code 
promulgated by its author precisely for use as legislation. 

Case law that derives from official incorporation of extrinsic 
standards is distinguishable in reasoning and result." 

A statute that refers to the law requires citizens to 
consult or use a copyrighted work in the process of fulfilling 
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their obligations. Your Honor, importantly, copyrighted works 
do not become law merely because the statute refers to them. 
Discussing referenced works or standards created by private 
groups other than incorporation by laws we have here, the Veeck 
court explains that to the extent incentives are relevant to 
existence of copyright protection, the authors in these cases 
deserve incentives. 

Now, my learned colleague brought up § 105 of the Copyright 
Act. I'd like to give you the reverse or other side of the coin 
to that, which is Copyright Act § 201 (e) : "No action by any 
governmental body purporting to seize, expropriate, transfer, or 
exercise rights of ownership with respect to the copyright, or 
any of the exclusive rights under a copyright, shall be given 
effect." 

So the mere incorporation by reference, as learned counsel 
said in Circular A-119, you have to be careful of the copyright. 
Also, we've already discussed th# CCC and Practice Management 
cases, much closer on the facts to this case. I will not go 
over them again. Your Honor, fair use. 

THE COURT: Again, I'm going to ask you the same 
question I asked Mr. Klaus, which would be, what would be a 
transformative use of your standards? 

MR. HUDIS: Mr. Klaus gave very good examples, and 
I will use them here; that is discussing, commenting on, 
critiquing our standards for one reason or another, and as a 
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matter of fact, that is done with our standards all the time. 

But it is not the wholesale copying, and it is not the wholesale 
copying and making available to the public for free of our 
standards which Public.Resource did here. 

THE COURT: And again, the same question. If you 
stopped selling the standards, are they still reasonably 
available under the OFR's regulation? 

MR. HUDIS: Three places. Your Honor. 

THE COURT: Libraries. 

MR. HUDIS: Thousands of libraries, the Department of 
Education, and the Federal Register office. So, yes, throughout 
the country. So all of the amici — you say, we need copies, we 
need copies? They've got them. So the nature of our standards, 
whether they are characterized as being core-expressive content, 
which they are, or assemblage of facts, which they're not, is 
usually not an important factor. 

The third factor. Public.Resource misappropriated our 
entire work, and Public.Resource's actions, as I've already 
explained to the Court, will drastically affect the market and 
value for plaintiffs' standards. It's just like in the ASTM 
case. This is a wholesale substitution for the purpose for 
which our clients promulgated these standards, making them 
valueless, at least in a copyright sense. 

Your Honor, some defenses also, in addition to fair use 
that Public.Resource raised for the first time, at least in our 
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case, in their briefs should not be countenanced by the Court, 
and that is the Copyright Act 102(b) systems process and 
procedure bar, the idea-expression merger doctrine, and the 
sense of fair doctrine, all of which, in any case, are 
inapplicable, as we briefed. 

Your Honor, there are two types of incorporation by 
reference defenses that we have here, one which was in their 
answer, the other one which was not, one which says, immediately 
upon being incorporation by reference, it becomes a fact. That 
was raised in their answer. We don't agree with that, and 
that's what the Court's going to decide. 

They're also saying that, by its very nature, these are all 
saying the same thing, these three defenses raised for the first 
time in their briefing, that it either is a system or process or 
procedure, it's an idea or expression, or it is scenes a faire. 

Your Honor, in each defense, there is no proof by expert 
testimony what is the idea, what is the expression, what is the 
system, and as a matter of fact. Your Honor, in our very 
standards, it says at the beginning, "evaluating the 
acceptability of a test or test application does not rest on the 
literal satisfaction of every standard in this document." 

THE COURT: You have to slow down again. We all speed 
up when we read. 

MR. HUDIS: You're right. Okay. 

"The acceptability cannot be determined by using a 
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checklist." This is at page 4 of our introduction of the 
standards, which is in evidence. 

"When testing an issue in legal proceedings and other 
venues, witness testimony, it is essential that professional 
judgment be based on the accepted corpus of knowledge in 
determining the relevance of particular standards in a given 
situation. The intent of the standards is to offer guidance for 
such judgments." 

THE COURT: But even without that preamble, Congress 
was aware of the potential issue that materials incorporated by 
reference posed when it crafted § 105. Ten years before then, 
it had given federal agencies the authority to incorporate 
private works, and it expressly stated that they would not lose 
copyright protection. So I'm not even sure that we need to go 
any further than that. 

MR. HUDIS: We would agree with you, Your Honor. 

THE COURT: I guess that question is more appropriately 
posed to defendants. 

MR. HUDIS: Yes. So, Your Honor, there's also other 
defenses that were raised. And by the way, none of these were 
briefed whatsoever. They should just be dismissed out of hand 
— just looking for it. Unclean hands, copyright misuse, and 
waiver and estoppel. We all — the plaintiffs move for summary 
judgment on that. Nothing was briefed by Public.Resource. 

All right. So, Your Honor, I think I am just about out of 
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time. Unless Your Honor has any further questions, I will save 
my remarks for the rest of the segments. 


THE 

COURT: Thank you, Mr. Hudis. 

MR. 

HUDIS: Thank you. Your Honor. 

THE 

COURT: All right. Whose going to — defendants. 


you have 45 minutes, obviously. Are you going to break it up or 


one person is 

going to do the duration? 

MS. 

MCSHERRY: We're going to break it up. Your Honor. 

THE 

COURT: All right. 

MS. 

MCSHERRY: I think we have 50 minutes, I hope. 


Perhaps we won't need it 


THE 

COURT: If says 45, but if you need to go to 50, 

I think we're 

okay. 

MS. 

MCSHERRY: Okay. 

THE 

COURT: Yes. 

MS . 

MCSHERRY: I'll try to keep my remarks as brief as 

possible. I, 

like everyone else, is very conscious of how much 


paper you've had to read. So I'm going to start with something 
surprising, which is that, for once, I agree with my colleague, 
opposing counsel, Kelly Klaus. This is a straightforward case. 

We think it's straightforward in a slightly different way, 
however. There are a lot of claims and defenses in this case, 
but I think it does boil down to one core issue, which is that 
the documents at issue here have been incorporated into law. 
That's why we're here, in essence. 
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THE COURT: Well, let's start with the last question 
I asked Mr. Hudis about. Hasn't Congress already ruled on this 
issue? And if copyright protection is going to be stripped from 
standards such as the one at issue here, isn't that something 
for Congress to decide to do and not this court? It does seem 
to be a matter of what the legislature wants. Copyright is not 
for me to — you know, I can't legislate copyright. 

MS. MCSHERRY: Sure. And I wouldn't ask you to. 

Let me take you back, if you would indulge me. I think we need 
to take this back to first principles a little bit before we 
decide what Congress is even allowed to do. We know what 
Congress legislated against was a background of 200 years of 
unbroken law that says that the law is not copyrighted. That 
much I think is not controversial. 

We have cases talking about opinions, cases talking about 
statutes, cases talking about regulations. In case after case, 
every court that's looked at his has said that the law is 
outside of copyright, and there's a reason for that: because the 
public has a fundamental due process and First Amendment right 
to access the law and to talk about the law, and those rights 
are sort of fundamental to self-government. 

THE COURT: But by what standard are you asking that 
I judge that the standards have enough creative expression to 
warrant copyright protection? What standard should I apply if 
deciding that? 
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MS. MCSHERRY: Well, I would suggest that you look to 
the BOCA case and you look to the Veeck case and look to the 
reasoning in both of those cases, and they looked at this issue 
in two different ways. 

First they looked at the tradition of case law that they 
had before them and came to the conclusion that, due to due 
process considerations in particular, the law was in the public 
domain. So that was the first part of the decisions in those 
cases. 

And to be clear, the BOCA case, what the BOCA case was 
doing was rejecting a preliminary injunction. But in the course 
of that — and then it remanded. But in the course of its 
rejection, it explained its reasons why it thought that the 
district court had got it wrong in holding that there was a 
possibility of success on the merits with respect to the 
copyrightability of codes. 

So it's a really — and I urge you to look to that case, 
because it's a very detailed explication of the tradition of 
case law that you also get in the Veeck case. But BOCA is 
earlier, and it's really one of the first cases to look at the 
problem of building codes and how we're going to look at them. 

I think it's also important to understand that all of the 
cases that have looked at this have looked at this one core 
problem, which is that we have a conflict between the exclusive 
rights that are granted to a copyright holder and our 
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constitutional rights to share the law and access the law. 

So the only place it got strange is, you know, we have this 
particular conundrum where we have this one area of law that 
operates a little bit differently because — and it's really an 
artifact of history. The Code of Federal Regulations was 
getting cumbersome. Yes. 

THE COURT; I just want to clarify something on the 
BOCA case that your mentioned, because you said that the district 
court granted the request for preliminary injunction. But when 
the First Circuit reversed that decision, it didn't do so based 
on the merits. 

MS. MCSHERRY: What it did is it remanded for further 
discussion. 

THE COURT: Right. 

MS. MCSHERRY: But it also spent quite a bit of time 
explaining why it thought the district court had got it wrong. 

So it seems to me that when we talk about a circuit split, 
we actually have a more substantial circuit split. It's not 
just Veeck versus Practice Management . 

We have Veeck and BOCA , and then, of course, we have the 
long tradition of cases that precede that. But these are the 
cases that most directly address our issue here, which is what 
happens when you've got standards that are incorporated into 
binding regulations and whether they're an exception to what is 
otherwise very clearly the rule* 
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THE COURT: But isn't your case made more difficult 
by the fact that you're not really asking — this is more of a 
case of a matter of ease of access. The codes and the standards 
at issue here are accessible without — you can look at them. 

You can read them. You can go make a copy of them at your 
public library if you need to if you don't have $22 to buy them. 

What you're asking for is to make them simply more easily 
accessible; right? It's not that they're not available; it's 
that they're not available as easily as you'd like them to be 
available; right? 

MS. MCSHERRY: Well, my client would certainly like to 
make them more accessible, but that's actually sort of a second 
point. The prior point is that if they are law, then of course 
we should make them more accessible as technology makes that 
possible. That's a wonderful thing. But either way, they're in 
the public domain. 

THE COURT: Well/ Congress considered this when it 
declared that simply by being incorporated, works didn't lose 
copyright protection, and one of the reasons is because of the 
public policy behind the creation of such standards, which is 
they want organizations to continue to promulgate such standards 
because they're for the public good. 

If they rob them of copyright protection, then there is no 
incentive to continue to promulgate these standards, and that 
was a factor that Congress took into consideration when it 
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declared that simply being incorporated by a reference didn't 
strip a work of its copyright protection. So I don't think the 
jump is as easy as you make it. 

You know, simply because it's been incorporated by a 
reference doesn't make it the law. It's been incorporated into 
certain laws, maybe, but the leap isn't quite that easy. And I 
guess that's where my concern is. What is it about these 
standards that you think make them the law? 

MS. MCSHERRY: Well, there's a couple things that I 
think make them the law. If you look at the IBR Handbook, for 
example, and you look to the National Archives website, which 
we've submitted to you, and in many, many other places there's 
an agreement that these standards, once incorporated by 
reference, have the force and effect of law. 

THE COURT: And? In other words, one key focus of the 
Ninth Circuit was whether there was evidence that individuals 
had been denied accesses to incorporated works. Have you put 
forth any evidence that anybody has been denied access, or are 
you saying that's irrelevant? 

MS. MCSHERRY: I actually don't think it's 
irrelevant. I think it's an important thing that distinguishes 
this case from Practice Management , because you're quite right. 
Practice Management says there's no realistic threat here of 
access to law, and if there were, that would raise due process 
and fair use issues. 
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THE COURT: And in BOCA , similarly, the government, 
the local government, anybody who wanted to see the building 
codes had to go buy a $22, or whatever it was, copy of the 
codes. That's not the case here. There is not just one place 
— you don't just have to have money to get access to these 
standards, and that's another key distinction between this case 
and BOCA . 

MS. MCSHERRY: So I think that the core question is 
what does copyright grant in terms of how you can condition 
access. So what we know is that, for example, one of the 
plaintiffs, the AERA plaintiffs, took the 1999 standards off the 
market altogether, until it came up in a deposition and they 
made them available again. 

The reading rooms that exist, you can only access them 
subject to after you sign a contract and give over your 
information, so it's subject to a lot of restrictions. And 
that's what happens when you allow folks to have a copyright 
in the law. What a copyright gives you, in any document, is a 
right to control and limit and restrict access, and that's the 
fundamental contradiction that — 

THE COURT: But in the case of these standards, it's 
not just that — there's only a certain amount of control that 
plaintiffs have. Once they're in the Office of Federal — the 
standards have to be available for viewing through the Office of 
the Federal Register; right? Plaintiffs can't just say, you 
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have to give us money to see these or you don't get them. There 
are other ways to get them. 

MS. MCSHERRY: So what the plaintiffs — what they're 
obligated to do currently is to simply deposit a couple of 
copies. So if you don't have the means to travel to Washington, 
D.C., and make a copy of the standards — 

THE COURT: Or go to a library? 

MS. MCSHERRY: Or if it happens to be in your local 
library, maybe it doesn't. And also, if you are print disabled, 
you're going to have a harder time getting access to these 
standards. And again, that's exactly what copyright confers. 
It's that statutory monopoly that lets you do that, and all of 
those restrictions are improper because they conflict with our 
constitutional due process and First Amendment rights. 

THE COURT: When Congress passed a National Technology 
Transfer and Advancement Act, it surely knew that the standards 
directed agencies to incorporate reference were copyrighted. 
Since the copyright protections are also statutory, wouldn't 
Congress have explicitly indicated that it was expanding the 
type of government works that cannot be copyrighted if it wanted 
to do that? 

MS. MCSHERRY: Well, I think that Congress didn't 
need to do that, for two different reasons. One is because we 
already had — well, two things. One is statutory right can 
never trump a constitutional right. So we'll take that as a 
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given. But secondly, the Copyright Act actually contains 
carve-outs — 

THE COURT: Right. 

MS. MCSHERRY: — for the law, the merger doctrine 

and 102(b), which both reflect this idea of the idea-expression 
dichotomy. And I would point you to a case that came later, 
but if you look to the case of Golan v. Holder , that's a Supreme 
Court case, and one of the things that that case says is when 
you have a tension between copyright and the First Amendment, 
we have certain doctrines that help resolve that tension. One 
of those is the idea-expression dichotomy. The other is fair 
use. And I would suggest to you that that's exactly what the 
Veeck court was up to. 

It recognized it had a constitutional tension, and it 
looked to merger, it looked to 102(b), to resolve that tension. 
The plaintiffs in this case talk a lot about constitutional 
avoidance, but I would submit to you that the Veeck approach 
and the BOCA approach are actually what gets you out of the 
Constitutional conundrum that you might otherwise have. 

THE COURT: You're asking this Court to balance the 
policy goal of unrestricted access to privately authored 
materials with a policy goal of providing continued incentives 
to private organizations to continue developing standards. 

Isn't that kind of balancing — didn't Congress already do 
that when it passed the Copyright Act and didn't list 
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incorporated by reference works among those that cannot have 
copyright protection under § 105? 

MS. MCSHERRY: Well, again, I would suggest to Your 
Honor that Congress didn't think it had to because it already 
had these carve-outs for the law, and it was legislating against 
200 years of case law, saying that the law was out of copyright. 
So they didn't need to reach this. 

The other thing that I would suggest is I do think this 
issue of incentives is quite important, and the plaintiffs talk 
a lot about this wonderful public-private partnership. And I 
don't disagree that there is a powerful partnership that happens 
here, but I think that it's false to suggest that no incentives 
will exist if the plaintiffs can't claim a copyright in works 
that have been incorporated into law. I think, to the contrary, 
they have tremendous incentives already. 

The fact that their documents are incorporated into law is 
very beneficial to them. They use it as a marketing tool 
because there's a — do I have... excuse me just a moment. 

I'll share with you just one example, if I may. This is 
an e-mail that NFPA sent out. It's an exhibit to our motion to 
strike, and it says, "Be confident that your electrical work 
complies with California law." So they know that the NEC, the 
National Electrical Code, has been incorporated into law, and 
they use this as a marketing tool. 

This is reflected also in the fact that when they write. 
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the NEC Style Manual specifically advises the folks who are 
working with it on how to write code-compliant regulations. 

They know their works are going to be incorporated into law. 

They benefit from their works being incorporated into law 
because it's a basis of other marketing. 

They also benefit because, as they said, and there's a lot 
of testimony about this, they want their works incorporated into 
law because that makes them mandatory, and they think that makes 
the world more safe. They may very well be right. 

THE COURT: The Fifth Circuit in Veeck said that, 
unlike model codes that are wholly adopted into law and impose 
legal obligations, these incorporated standards — and I guess 
that's where the plaintiffs assert that they differ from Veeck 
— these incorporated standards are only required to be 
consulted or used in the course of fulfilling existing legal 
obligations. They're not binding law. 

So isn't that what the cases here — I mean, Veeck drew 
that distinction, and don't plaintiffs fall on the other side of 
that distinction? In other words, the standards at issue here 
have been incorporated, but they themselves don't — in other 
words, plaintiffs can't send out e-mails saying if you don't 
follow our codes or our standards, you're falling afoul of the 
law. They can only say to the extent they're being -- they're 
not like building codes or model penal codes or commercial 
codes; right? 
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MS. MCSHERRY: I would disagree with you. Your Honor. 
If I build a building and it doesn't comply with the National 
Electrical Code, I'm going to face penalties. If I don't comply 
with a national fire safety code — the various ones, there are 
many — I'm going to face penalties. But also, if I'm a parent 
and I want to know if the school that my child goes to is 
complying with fire safety regulations, I want to know what 
those fire safety regulations are because it's supposed to be 
built to that code. That's what incorporation by reference 
means. It means it has the force and effect of the law. 

THE COURT: Once it's incorporated. 

MS. MCSHERRY: Once it's incorporated. That's 
correct. One other thing I'd like to speak to is this issue of 
Veeck and intent. So, first of all, I'd like to just clarify 
that the Veeck holding was based on two separate grounds. 

The first part of the Veeck holding, the Veeck court looks 
at the Banks cases and concludes that the due process 
considerations there apply with respect to model codes as well. 

But the issue of intent. So the Veeck court's merger 
analysis does not depend on intent. The Veeck court's merger 
analysis depends on its conclusion that, once incorporated by 
reference into law, the expression and the idea merge. There 
is no other way to describe what you have to comply with. Just 
like the Constitution, just like the tax code, the Code of 
Federal Regulations works the same way. 
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THE COURT: The standards here that are incorporated 
by reference provide guidelines and procedures in some of them 
that individuals or entities have to use or reference in 
fulfillment of their legal obligations under federal 
regulations. But again, and I think this is a significant 
difference, there's no evidence that anyone here has been denied 
access to the standards. What you're arguing is that people 
should have better access to the standards. That wasn't the 
case in Veeck, was it? 

MS. MCSHERRY: So what I'm arguing is that the law is 
not copyrightable, and, therefore, as technology develops, we 
can make access better and better and better. Access comes 
second. Access is important, but it is not the only thing. 

THE COURT: Some of the standards that have been 
presented to me, for example, ASTM, the 86-07, which is at page 
107 and 6, include what a law review article refers to as 
secondary references where to fully comply with the standard you 
also have to comply with a list of other standards. So what's 
your position on whether these secondarily referenced 
standards — have those also lost copyright protection? 

MS. MCSHERRY: So I think what — 

THE COURT: Even if they're incorporated into the 
incorporated standard or they're included in the incorporated 
standard? 

MS. MCSHERRY: Where does it end? 
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THE COURT: Yeah. 

MS. MCSHERRY: So I think where it ends is I would go 
back to the CFR, to the Code of Federal Regulations, and ask 
what has explicitly incorporated there, which is what we're 
presented with here. 

Now, if there's further references on top of that that 
aren't explicitly incorporated, I think we might understand that 
differently, and in any event, my client doesn't publish those. 
He's trying to publish and create a sort of grand, unified CFR, 
because what we have right now is a very disjointed Code of 
Federal Regulations where we have sort of one code of 
regulations that's online that you can see. But then it refers 
out to hundreds of other standards that you then have to 
separately consult if you want to understand what the law is. 
That's the core of our problem. 

I'd like to talk a few minutes about — well, I think I 
want to answer a question that I think you were asking earlier 
about Veeck's focus, also on intent, and that building codes, 
the model codes in that case, were intended to be created into 
law. I think I've already referred to this, but I would say 
this again. There's ample evidence in the record that the 
standards organizations know very well and very much want their 
standards to be incorporated into law. 

THE COURT: And? I mean, of course. If you 
promulgate standards and you sell them, isn't it better for you 
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if your standards are promulgated into law because more people 
will want to buy them? Does that rob them of copyright 
protection, the fact that they hope that some governing bodies 
or some local governments or federal governments will incorporate 
their standards? Doesn't that mean they've been successful? 

MS. MCSHERRY: Well, I think what it just speaks to is 
this question that I think the plaintiffs have tried to suggest, 
that Veeck turns on the intent of the creator, and I'm just 
simply trying to answer that question — 

THE COURT: And their intent is? 

MS. MCSHERRY: Their intent is to have them made into 
law, and that's fine. Again, I have no quarrel with that, and I 
think having stuff being incorporated into law is a tremendous 
marketing tool. But it also helps make us all safer. We don't 
quarrel with that either. 

What we quarrel with is the proposition that once one has 
accomplished that goal of incorporation into the law, somehow 
you still get to control and restrict access forever. We have a 
plaintiff, again, who took one of the standards off the market 
altogether. And the reading rooms that exist, they exist now. 

They may or may not exist tomorrow. 

THE COURT: But isn't the solution to that issue the 
responsibility of Congress? I mean, if Congress wanted to strip 
materials incorporated by reference of all copyright protection, 
they could do so very easily and very clearly. And your argument. 
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well, they didn't need to do that in this case is — you know, 
nobody wants to try to figure out what's in the mind of Congress 
when they do something, but when they have the power to enact or 
to declare what's covered by a copyright or not, they do so. 

The fact that they explicitly left works incorporated by 
reference with copyright protection means that you want me to 
now say, well. Congress, I know you said that they have 
copyright protection, but, actually, under these circumstances, 
they don't. And isn't that action one that's really meant for 
the legislature? 

MS. MCSHERRY: I don't think so. Your Honor. For one 
thing, I don't think that Congress can make an unconstitutional 
bargain, and so if there are, as we believe, the fundamental due 
process and free speech considerations in play here. Congress 
can't write a statute — 

THE COURT: Copyright protection comes from the 
Constitution as well. I mean — 

MS. MCSHERRY: Copyright protection is — sorry. 

THE COURT: It is of constitutional dimension, and 
therefore — if we're talking about what the framers wanted 
in district court, we're in trouble. One could argue that 
copyright, having derived from the Constitution, that Congress 
is well aware of what it can do and not within the Constitution 
even in the face of the Due Process Clause. 

MS. MCSHERRY: I completely agree with you. 
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THE COURT: Is there a case you can cite to me where 
a court has done what you're asking me to do where the standards 
were available? Not where the standards had to be purchased, 
but where someone without funds could access the standards. 

MS. MCSHERRY: So, actually, I think in the Veeck 
case, if you wanted to go get hold of them and you wanted to go 
to this little town, the person who posted the standards online 
was able to acquire them. So you can get hold of the standards. 
But again, I want to reemphasize that this case does not turn 
simply on accessibility. That's just a benefit of it. 

THE COURT: Right. Because you're saying that the 
standards were already basically not capable of being copyrighted 
once they were incorporated by reference. 

MS. MCSHERRY: That's correct. And, Your Honor, I 
would say to your earlier question, of course copyright derives 
from the Constitution as well. But nonetheless, it's very clear 
that copyright is a statutory right, and statutory rights don't 
trump constitutional rights. 

THE COURT: Can you cite me a case where a court has 
said that regardless of their accessibility, once a standard has 
been incorporated by reference into a law, it loses copyright 
protection? 

MS. MCSHERRY: I think that's exactly what the Veeck 
case is saying. I think that's what that case is saying, and I 
think it's what the BOCA case is saying. And they're saying it 
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against a background of hundreds of years of case law. 

I'm mindful of my time, and I want to make sure I leave 
time for the remaining issues, so I just want to touch on a 
couple of other issues. 

One is with Practice Management . Again, Practice Management 
said there was no realistic threat of access. I think we do 
have that here. I don't think the case turns on that, but it 
does acknowledge that if there were such a threat, they would be 
more concerned about due process. But that evidence is simply 
not before the Court. 

The other thing that Practice Management was worried about 
and CCC was worried about is depriving the SDOs of incentives, 
and as I think we've discussed, there are plenty of incentives 
that would still exist. 

The final thing I want to speak to is the issue of takings, 
because there's been sort of a lot of hand-waving around about 
maybe creating a takings problem. 

THE COURT: Well, I want to ask you, what about — in 
its Notice of Proposed Rulemaking, OFR relied on your argument 
— well, it addressed your argument, and it ultimately rejected 
a proposal to require free online access to standards in its 
"reasonably available" determination. 

It said, "If we required that all materials IBR'd into CFR 
be available for free, that requirement would compromise the 
ability of regulators to rely on voluntary consensus standards, 
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possibly requiring them to create their own standards which is 
contrary to the NTTAA and the OMB Circular A-119." 

Doesn't that indicate a congressional intent to continue 
to give copyright protection for standards incorporated by 
reference? 

MS. MCSHERRY: I think the OFR came to that conclusion 
because the SDOs came and said the exact same thing they're 
saying here, which is we'll take our toys and go home if we're 
not allowed to have copyright protection. 

THE COURT: But isn't that factor perfectly reasonable 
for Congress to consider? In other words, the Congress can say, 
look, if we strip these standards of copyright protection, 
there's not going to be any more of this voluntary consensus 
standard development, and we're going to have to — it's going 
to be a problem for the government. So, in return for that, 
we're going to allow them to continue to keep their copyright 
protection. Isn't that something that Congress is allowed to 
do? 

MS. MCSHERRY: Congress could do that, but I don't 
think that's actually what Congress did. 

Now, what the CFR said, it went through a lot of the 
arguments, and it said we think it's beyond our authority to 
do what the petitioners, including my client but not just my 
client, want us to do. We think it's beyond our authority to 
interpret reasonable availability in the way you want to. 
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We think that it will cause problems for the agencies in 
terms of monitoring compliance. So they had various concerns, 
but those concerns don't apply here, because what we have here 
is my client who's willing to make these standards available 
right now, very easily, and it doesn't depend on any agency 
action whatsoever. 

Just two final points. Again, with respect to the takings 
question, what I would like to say about that is, in addition to 
the fact that I don't think it's a credible concern given the 
tremendous benefits of incorporation by reference, aside from 
the ability to sell the standards — which, by the way, most of 
the standards aren't much used anymore anyway except for as law. 

But the other thing that I think we can say with respect to 
takings is that essentially that's a different process. In the 
Veeck case, in the wake of the Veeck case, we didn't see a 
takings claim, and if the standards development organizations 
want to try to bring a takings claim, which I think, again, is 
unlikely, if they were to bring it, that's a whole separate set 
of facts to present to the court. 

THE COURT: Let me ask you a question regarding the 
merger analysis. 

MS. MCSHERRY: Sure. 

THE COURT: Could I find that the standards lost 
copyright protection under the merger doctrine but not find that 
they've lost protection by becoming law? Could I do both those 
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things ? 

MS. MCSHERRY: I think that — so the — you mean once 
they've been incorporated by reference? 

THE COURT: Right. In other words, could I find that 
they retain their protection by becoming the law, but they lose 
protection under the merger doctrine? 

MS. MCSHERRY: I think that you have to say that they 
lose protection under the merger doctrine because they become 
ideas, and the idea and the expression merge. Essentially, they 
become facts. 

THE COURT: Okay. Is your merger approach a separate 
theory or just a subpart of your public domain theory? Because 
it wasn't clear to me. 

MS. MCSHERRY: Okay. I tend to think they go 
together. The way that I conceive of them is that the first is 
really I think the way the Veeck court tried to conceive of it, 
which is first we have our due process concerns. And following 
that case law, we have to say that anything that's been 
incorporated into law, made regulation, is out of copyright; and 
so Veeck could make a copy of the law — and the court stresses 
that at 800 — could make a copy of the law under Banks and 
related cases. 

But then the second portion of the analysis is to then look 
to the Copyright Act and see if there's a way to reconcile that 
fact with what already exists in the Copyright Act. So the 
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Veeck court turns to the merger doctrine and says, in addition, 
even if — the quote is, even if Banks fails, I can still look 
to merger to find that these model codes have been incorporated 
by reference into law, and therefore the idea and expression 
have merged. They're facts like the tax code, like the 
Constitution. 

If you don't have further questions — sorry. You do. 

THE COURT: Well, the scenes a faire doctrine, I have 
to confess I'm not quite sure how it's applicable here. Are you 
arguing that if somebody tried to write their own standards on 
the exact topic as one of the standards here, they would still 
have to be identical down to the word choice and the punctuation? 
Is that my understanding? I was a little confused by your 
argument on this. 

MS. MCSHERRY: So that argument in particular goes to 
the copyrightability of the standards as such, and our argument 
is that if you look at how they're created, they're very much 
shaped by external factors that are external to the sort of 
creativity of anyone involved in drafting them. 

THE COURT: Okay. All right. 

MS. MCSHERRY: Okay. Thank you. Your Honor. 

THE COURT: Thank you. 

Oh, I'm sorry. My court reporter needs a break. He's been 
going for — and we're running behind. We just keep plowing 
along. 
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(Recess from 10:44 a.m. to 10:54 a.m.) 

MR. BRIDGES: Good morning, Your Honor. 

THE COURT: Good morning. 

MR. BRIDGES: I'm Andrew Bridges, also representing 
Public.Resource. 

THE COURT: Good morning. 

MR. BRIDGES: And I will address fair-use issues, 
which are vitally important to the case. Before I get to my 
statements that I'd like to make — 

THE COURT: Oh, and I have pushed my meeting to 1:00, 
which means we're only five minutes behind instead of half an 
hour or something. 

MR. BRIDGES: Thank you. Your Honor. 

Before I get to my own point, I wanted to address something 
that Mr. Klaus said on the other side: No case in the history 
of fair use has endorsed an entire work being made available 
widely for download or distribution. 

Well, I'd like to call the Court's attention to a number 
of cases that did exactly that. Important cases. Cases from 
various United States Courts of Appeal. 

I refer to the Court to Nunez v. Caribbean International , 
First Circuit. Full copies of original pictures of a model 
were widely disseminated by a newspaper when it became 
newsworthy that this model, who had some racy photos, had become 
Miss Universe Puerto Rico. The First Circuit found fair use 
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from that widespread publication of the full photos. 

The Second Circuit, in Swatch Group v. Bloomberg , found 
fair use the widespread online dissemination of materials from 
investor conferences that Swatch Group claimed a copyright in. 

THE COURT: But the Swatch case in particular, that 
was the case where the conference call was closed, and without 
the dissemination of the materials, the materials would not have 
otherwise have been accessible, the information. 

MR. BRIDGES: That's a different point. Your Honor. 

What Mr. Klaus said is there is no point in the history of fair 
use where an entire work was disseminated broadly to the public. 
His point was an entire work plus public dissemination. It 
wasn't about the nature of the original work or the circumstances 
of the original work. 

But to address your issue, the Ninth Circuit in 
Hustler v. Moral Majority , where Larry Flint had basically sent 
up Jerry Falwell in Hustler magazine, and Moral Majority, only 
bleeping out some obscene or offensive words, disseminated 
widely for fundraising purposes the entire item featuring 
Mr. Falwell. 

Righthaven v. Jama . Now, I've given you appellate cases, 
but there's also an important case out of the District of 
Nevada, 2011. Righthaven v. Jama found fair use in the 
widespread public dissemination of an entire article from the 
Las Vegas newspaper. 
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So the notion that fair use doesn't allow widespread 
dissemination of an entire work is simply wrong, and Mr. Klaus 
referred to the HathiTrust decision in the Second Circuit 
because that case does talk about certain security features that 
HathiTrust imposed. But that's not necessary. That was 
incidental to that one decision, and it's wrong to ignore all of 
these decisions that do allow entire works broadly disseminated. 

THE COURT: And that may be, but how is that germane 
to this discussion here? In this case, there's no evidence that 
has been proffered that the standards at issue weren't otherwise 
available. I can definitely see a fair-use argument being made 
for a situation in which, absent the fair use of the material, 
the information would not otherwise be accessible. 

MR. BRIDGES: Your Honor, whether they are otherwise 
available actually doesn't make a defense to fair use at all. 

It really doesn't. And I'll go through the standards. I just 
wanted to address the cases — 

THE COURT: So is it your position that — where's the 
line drawn? I can — you know, if there's a book coming out, 
the latest Harry Potter book is coming out and it's copyrighted, 
can you download the entire book and make it available to the 
public? No. 

MR. BRIDGES: Likely, no. And that's nothing close to 
our argument. I think it might be helpful if I go through the 
factors. I just wanted to rebut the point that Mr. Klaus had 
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made that there had been no case in the history of fair use 
about entire works being disseminated. 

THE COURT: That's less important to me. All right. 

MR. BRIDGES: So let me just explain. First of all, 

I think the parties agree that fair use is amenable to summary 
judgment, and we have summary judgment in Nunez and Authors 
Guild v. Google . It's important to understand that fair use is 
outside the statutory monopoly of copyright. 

Section 106 gives the rights of the copyright author, and 
the section starts with the wording, "Subject to § 107." That's 
fair use. Section 107 states fair use. 

It says, "Notwithstanding the provisions of § 106, fair use 
is not an infringement of copyright." There's a boundary zone 
between the rights of the author and fair use. Fair use, 
therefore, takes nothing away from a copyright holder because 
the rights of a copyright holder don't extend into fair use 
anyway. 

THE COURT: How is downloading a set of copyrighted 
standards in their entirety and placing them on the Internet for 
free fair use under the definition of fair use as I have it? 

MR. BRIDGES: Well, Your Honor, to begin with, 
let's talk about the structure of fair use in the statute. 

The statute says there are four factors to be taken into 
account; and it specifies the factors, and I will go through 
them. Campbell v. Acuff-Rose also explains that the task of a 
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court is to analyze all four of those factors in light of the 
constitutional purpose of copyright, which is to promote the 
progress of science in the useful arts. 

So let's go through those factors, and I will say this. 
You've heard about some constitutional issues. As the Supreme 
Court has said, fair use as a doctrine brings First Amendment 
considerations into the Copyright Act. It has built-in First 
Amendment accommodations. 

So the first nonexclusive statutory factor -- let me back 
up. Section 107 gives the four factors. It also gives several 
examples of paradigmatic fair use in the introduction to the 
section. It says, "Fair use, including" and it has several 
examples, "is not an infringement." And then it gives the 
factors. 

The first factor is the purpose and character of the use. 
This is the defendant's purpose and character of the use, and 
the purpose and character of Public.Resource's use is for a 
very, very important public benefit. It is to report the law. 

It says what the law is. When you saw that certificate that 
Public.Resource distributes, that is underscoring — it's making 
a political point. It says. This is law. 

THE COURT: But the point of the matter is, this law 
as you call it, these standards, are available in libraries. 
They're available in the Office of the Federal Register. 

They're available in reading-room online sites. What you're 
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doing is making the standards available for downloading by 
someone who, for example, could download the standards and sell 
them; right? 

MR. BRIDGES: That is not the purpose. 

THE COURT: Right. You have purpose, and then you 
have reality. And Congress decided that, and the framers — and 
we're back to the framers — decided that copyright existed to 
give the benefits of ownership to people who created material so 
that people would continue to create material. 

Congress decided not to strip copyright protection for all 
material that was referenced by law, for that same reason, 
because, otherwise, people would stop promulgating these 
standards or people would stop promulgating whatever it was that 
was being incorporated by reference. 

But what you're saying is, because our purpose is noble and 
good, then it's fair use. The problem is, your purpose may be 
noble and good, but despite that, you are stripping the 
creators, the owners of the copyrighted material, of commercial 
use of their product. 

MR. BRIDGES: Your Honor, the Supreme Court did 
exactly that. It incorporated the full lyrics of "Pretty Woman" 
in the opinion of Campbell v. Acuff-Rose , and if my purpose is 
to distribute copies of that opinion and some people use it to 
get access to the lyrics of that song, well, that wasn't my 
purpose. It's not chargeable to me. But the Supreme Court put 
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the full lyrics in its opinion, and I'm allowed to have my 
purpose. 

THE COURT: But the Supreme Court, somewhere in there 
there was an opinion. The lyrics of the song were part of the 
opinion, but the purpose of that publication of the lyrics was 
because they were involved in a Supreme Court opinion. You're 
not doing anything but lifting these standards wholesale and 
putting them on a website. 

MR. BRIDGES: Your Honor, that comes to the third 
factor of fair use, and I will go there. Well, actually, the 
third factor, as I think Campbell says and as HathiTrust says, 
the third factor on amount and substantiality of use depends on 
the first factor, what the purpose is. 

The third factor, the amount, depends on the purpose. And 
what's the purpose here? It's to report the law. That's where 
all the focus has been. The purpose of the defendant is also to 
make the law amenable to research and scholarship. 

One can do textual analysis, data analysis on these that is 
not available in any other way. That's why these were 
reformatted into HTML. They are word searchable by the public 
in a way that the reading rooms can't be done. The reading rooms. 
Your Honor, they've got a document that basically talks about 
how they're making the reading rooms inconvenient. That's their 
purpose, is to make it inconvenient so that they can sell it. 

Public.Resource's purpose is to make the law available to 
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the public, and there is no other way to make the law available 
to the public than by presenting the law itself. It is a 
factor. It goes to the merger point Your Honor made earlier. 

When something becomes the law, that text is now a fact. 

It is the law. So Public.Resource is getting these re-keyed so 
that they are text searchable and so that they are accessible to 
the blind. It wasn't the sole purpose by any means, but it's 
something that the plaintiffs haven't done because of what the 
defendant has done. 

THE COURT: Public.Resource started doing that after 
this lawsuit was filed, didn't it? 

MR. BRIDGES: No. I believe it was done beforehand. 
Your Honor, and it's been part of the process. So the purpose 
is to facilitate research and scholarship. The purpose is to 
foster inclusive access for persons to this. 

Now, the purpose is also noncommercial. Public.Resource 
is not trying to go into competition with the plaintiffs. 
Remember that the only standards that Public.Resource has acted 
on are standards that have become law. This is not about 
competing with the thousands of standards that they do. This is 
about 250 standards, roughly. 

Commerciality does enter into the first factor of purpose 
and character here, and Campbell v. Acuff-Rose , that was 
commercial. The Supreme Court endorsed it. Swatch Group v. 
Bloomberg was highly commercial. The Second Circuit endorsed 
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it. Nunez v. Caribbean International , highly commercial. 

The First Circuit endorsed it. 

I'd like to go to another important aspect of the purpose 
and character of the use, and that's the transformative use. 
What's important here is that transformative use means a new and 
different use or purpose. It does not mean that the work has to 
be different. In all the cases I've been discussing up to now, 
there was no change in the work itself, but the original work 
was used for a new and different purpose. 

For example, there's a Fourth Circuit case, Bond v. Blum , 
where one party in a child custody case took an entire 
autobiographical manuscript of one of the parties and put it 
before the Court. It was a different purpose because that was a 
fact. 

Now, here's an interesting question, Your Honor. 

I think the other side has skirted the issue. Let's match our 
purpose. Public.Resource's purpose, to the plaintiffs' purpose 
in creating their standards. Was the plaintiffs' purpose to 
write law? If their purpose was to write law, then we have a 
similarity of purpose, and if their purpose was to write law, 
then they're falling into deeper and deeper Veeck and BOCA 
problems. 

But if, as they say, oh, but we had all these purposes 
that had nothing to do with the law, we had best-practices 
purposes, we had contractor purposes, then the law purposes of 
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Public.Resource are very different, and that's an important 
point here. They are not competing. These purposes are very, 
very different. 

THE COURT: What's the line between transformative and 
not transformative here? I mean, if you had converted the hard¬ 
copy standards to a searchable PDF but had only posted on your 
website that it was available for free upon request, would that 
have been transformative? 

MR. BRIDGES: Your Honor, it's transformative 
because it is for a different purpose and a different use. 

The conditions of that use don't affect the issue. It was a 
different purpose, a different use. 

THE COURT: If the PDF versions that plaintiffs sold 
were also searchable — in other words, if plaintiffs sold a 
searchable PDF version, is the only transformative aspect of 
your posted PDF standards the cost, that it's free? 

MR. BRIDGES: No, Your Honor. I have to say very 
clearly: different use, different purpose to make the law 
available. 

THE COURT: I understand that. I understand that. 

I'm asking with regard to the transformative-use issue. Putting 
aside the purpose, if you said you can get this if you ask for 
it, or if plaintiff also offered what you're offering but it 
cost money, isn't the law being reported? It's not just 
reporting the law that you want to do. You want to do reporting 
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the law for free; right? Because the law is free. 

MR. BRIDGES: Yes. Absolutely. 

THE COURT: Right. 

MR. BRIDGES: Because we believe that no private 
party should be exercising a private monopoly over the law, and 
it is not just about seeing the law; it is about speaking the 
law. It is about analyzing the law. It is about critiquing. 

They said critiques can be transformative. Great. 

Critiques can be transformative only if you have access to be 
able to critique them. They're saying you have to pay them to 
critique them, or you have to maybe go to one or two places in 
the United States. And by the way, the statistics that AERA 
gave you about library access — 

THE COURT: Right. We're running behind. 

MR. BRIDGES: All right, if I can get back. The point 
is, part of the purpose here is to facilitate public discourse 
about the law without people having to pay a toll in order to 
know what the law is or without having to go to Washington, 

D.C., to get access or to have to pay them $49 to know what the 
law is in order to critique it. 

There's a very, very important political point here, that 
there should not be — in this public-private partnership that 
they have discussed, there should not be private dominion over 
public law. 

THE COURT: And there's a very big, white marble 
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building about two blocks away where you make those political 
points, not in the district courts. 

MR. BRIDGES: I know, Your Honor. 

THE COURT: Aren't you just in the wrong forum for 
that point? 

MR. BRIDGES: Absolutely not. This is exactly the 
right point. This is the right place for the fair-use argument, 
because Congress set factors precisely for courts to use. It's 
a flexible doctrine for courts to analyze on a case-by-case 
basis. That is what § 107 is. 

It says, "Here you go, courts. Here's the standard. Have 
at it." And there is a rich, rich jurisprudence of judge-made, 
fair-use law that is understood to be the proper dominion of the 
courts. That's why we're talking about fair use. Your point is 
a different point about the determination of copyrightability. 
But when it comes to fair use, courts are the very, very center 
of that focus. 

I need to talk, though, because you are concerned about 
some of the substitution effect. Actually, before I get there, 

I want to get to the second factor, and that is the nature of 
the copyrighted work. 

Now, the nature of the copyrighted work, when it is adopted 
for dissemination by Public.Resource, at this point it is the 
law. This is not merely — this is not merely some building 
best practice. The nature of the work, when it enters into 
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Public.Resource's world, it is the law. It is the fact of law. 

So Public.Resource is reporting facts, and these are things that 
had been publicly disseminated to the public. Okay? That 
actually weighs in favor of fair use, not against fair use. 

Harper & Row , there was no fair use because a private, 
nonpublic manuscript was purloined. The Mange case was private 
wedding pictures that were purloined. The fact that they were 
publicly available weighs in favor of fair use because there's 
no preemption of the first publication availability. That 
weighed on the court in Harper & Row . 

I must say this. Your Honor: The works that are on PRO'S 
website, Public.Resource's website, almost all of them — it may 
be one or three or four out of maybe 250 — have been superseded 
for their purposes. They are not the current standards. They 
are still the law. That's why it matters to Public.Resource. 

They are still the law, but they are not their current standards. 

So Public.Resource isn't interested in their standards as 
standards. Public.Resource is interested in the law. So this 
is a huge point that the second factor, the nature of the 
copyrighted work, is in this case — they are obsolete or 
obsolescent standards, by their standards, but the nature of the 
copyrighted work insofar as Public.Resource is interested in it 
is because it's still the law. 

THE COURT: But once the 2014 standards become 
incorporated by reference, you're going to want to put those up 
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as well; right? 

MR. BRIDGES: Yes. All the same reasons, and for 
salutary reasons. It's entirely appropriate. I would also like 
to discuss — the third factor is the amount and substantiality 
of the work compared to the original, yet it does turn on what 
the purpose is. 

Again, at the beginning of my time I gave the Court five or 
six cases, most of them from circuit courts, where the entire 
work was used. That doesn't weigh against fair use when the 
purpose is to present the law as law. 

There is no way of saying, well, we'll give you a summary 
of the law. People don't have to obey a summary. There was one 
executive — I've forgotten the company. One prominent executive 
went to prison for violating a standard that was incorporated by 
reference. Went to prison. If you're trying to make public 
what the law is, you have to give the whole thing. 

Finally, I do want to talk about the fourth factor, which 
is the effect of the use on the potential market for or value of 
the copyrighted work, and this is where I think they are saying, 
oh, look, we're going to lose business. You're concerned that 
they're going to lose business. 

First of all, this factor focuses on loss to the copyright 
value, not losses to other values. The factor must focus on the 
standards at issue in this case. What's interesting is when 
they use some experts to try to talk about substitutive effect, 
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for reasons we can just talk about in motions to strike, the 
experts shot air balls with extraordinary mistakes. 

For example, Mr. Geisinger for AERA attributes the decline 
of sale of standards to Public.Resource, missing the fact that 
the catastrophic decline that he's looking at began a year, year 
and a half before Public.Resource ever posted anything. 

As a matter of fact, the sale of the standards appeared 
to go back up towards the end of the time that Public.Resource 
had it up there. There is no real evidence of the loss. And 
when they talk about the harm, they talk about loss of control. 
They don't have real numbers about any substitution effect. 

They don't. 

THE COURT: Well, are you really arguing that it's not 
rational to conclude that if their standards are available for 
free for anyone to download off the Internet that people aren't 
going to buy them? That's a logical conclusion, isn't it? 

MR. BRIDGES: No, Your Honor. It's a speculative 
conclusion, exactly the sort of speculative conclusion that the 
Supreme Court rejected in the Sony Betamax case. The argument 
that, oh, people are going to stop watching live TV and they're 
going to stop watching movies because of the Betamax, and the 
Supreme Court expressly rejected that as speculative. 

And we have ASTM's president, Mr. Thomas, stating that we 
have seen no measurable effect from Public.Resource's actions. 

We have seen no measurable effect, and they have substituted 
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hypothesis, conjecture. 

The point is, what is expanding is access. Yes, there are 
accesses to these. That's very good, because that means that 
more people are seeing, reading, speaking, analyzing the law. 

More access is a good thing. They have not shown any competent 
evidence of actual losses, and we have ASTM's president 
admitting no measurable effect. 

Your Honor, I think I'd like to say one — 

THE COURT: You need to make it brief. 

MR. BRIDGES: — more thing. That's right. 

I would like to come back, however — we've got the four 
factors in fair use, and it is the Court's province, emphatically 
the Court's province on fair use. That's why we have all these 
fair-use cases. People could have argued in all of those cases 
that Congress could have adjusted copyright law, but Congress 
has expressly given the courts authority over fair use because 
it's an equitable case-by-case doctrine. 

But as Campbell v. Acuff-Rose made clear in the Supreme 
Court, it's the job of the courts to analyze the four factors in 
light of the constitutional purpose of copyright, which is to 
promote the progress of science in the useful arts. 

To promote the progress of science in the useful arts means, 
in the case of law, the study of law, the critique of law, and 
the education about the law, giving full public access to the 
law and ruling that whatever statutory monopoly they have over 
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their building standards, they do not have a private monopoly 
over the law. We have this important carve-out. It's a 
statutory boundary between the rights of the copyright holder 
and fair use. 

So we ask Your Honor to look at these factors and to 
understand that this purpose is a laudable and appropriate 
purpose. The nature of the work is as factual as it could be. 

It is the law. Your Honor could rule that it is merged; it is 
fact. You could rule that there's no copyright at all. But 
fair use allows a pressure valve here. If the Court is 
uncomfortable ruling that it's not copyrighted, fair use is 
exactly how to accommodate the concerns on both sides. 

Thank you. Your Honor. 

MR. HUDIS: Your Honor, we did reserve some time for 
rebuttal. I will take less than five minutes. 

THE COURT: All right. 

MR. HUDIS: Your Honor, I'll just take the issues that 
are of most concern from the presentations from Public.Resource. 
First, with reference to the BOCA case at page 736, in remanding 
the case for further argument after reversing the preliminary 
injunction, the case says, "The rule denying copyright 
protection to judicial opinions and statutes grew out of a much 
different set of circumstances than to these technical 
regulatory codes." 

All right. As to our standards being off sale for a time. 
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as we discussed, Your Honor, they were still available in 
thousands of libraries, and if one could not get it from one 
library, there's an inter-lending program between libraries. 

Your Honor, Public.Resource is asking this Court to 
substitute its judgment for the will of Congress. Mr. Bridges 
spoke about one of the exceptions to copyright. There are a 
number of exceptions to copyrighting, sections 107 through 121 
of the Act, and Congress, through all of this, has not seen fit 
for a special exception to copyright that Public.Resource now 
would like to introduce. 

As to the external factors in creativity, in their 
briefs and in responses to our statement of material facts. 
Public.Resource has already conceded that we have copyrightable 
content in our book. The HathiTrust case, the central holding 
of that case was to guard against entire dissemination essential 
to the court's decision. 

Mr. Bridges brings up the fact that HTML and OCR coding 
were done of the standards. Not in our case. It just went up 
as a standard graphic PDF. 

Now, you asked about the dividing line between what is and 
what is not transformative. Your Honor, if you could look to 
the Leval article where all of this transformative language 
originated, cited by the court in Campbell v. Acuff-Rose , it 
says the mere repackaging and republishing of the original does 
not pass that test. 
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And finally, as to the alleged obsolescence of our 
standards, Your Honor, those standards are still valuable today 
for any test that was promulgated between 1999 and 2014, and 
those standards are still applicable today. They are still on 
sale today, and what Public.Resource is doing would endanger our 
income to further promulgate standards in the future. 

Thank you. Your Honor. 

THE COURT: Thank you, Mr. Hudis. All right. 

MR. KLAUS: Thank you. Your Honor. 

Mr. Bridges misheard me on fair use, because I did not say 
there's never been a case in the history of fair use that has 
not said that the copying of a work — a work — would not be 
fair use. 

What I did say was that there's never been a case in the 
history of fair use that has said setting up an entire business 
of the repeated copying and distribution of entire works would 
be fair use. And, in fact, the Authors Guild v. Google and 
Authors Guild v. HathiTrust case made it clear that would not be 
acceptable. 

Mr. Bridges also said there's no evidence of actual 
substitution, actual market harm. I would simply give cites 
to Your Honor to places in the record. Mr. Berry's declaration, 
paragraphs 11 through 12, which talk about people disseminating 
entire PDF copies of the works. Mr. Bridges also said that 
Public.Resource alone makes the works available in HTML or text- 
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searchable format. 

In fact, if you look at Mr. Thomas's declaration at 
paragraph 44, what he says is that they actually make their 
standards available in text-searchable format. The difference 
is — as does my client, NFPA. The difference is that if 
somebody wants that, that's a different format that they pay the 
right for. 

Finally, I'd like to go back to Ms. McSherry's point on the 
Veeck case. Two things to note about it. One is an entire 
section of that that talks about the difference between model 
codes and extrinsic standards. I've discussed why I think the 
"sole purpose" language, which is the qualifier which the Veeck 
court, which the defendant is relying on, put on to that 
distinction. 

I would also point out that that was in response — that 
entire discussion in Veeck was in response to amici filings, 
not just by anyone, but by my client, by ASHRAE. That was the 
qualification that the Court put on. 

Happy to answer any other questions. 

THE COURT: Thank you. 

MR. KLAUS: Otherwise, I'll just move on. Your Honor. 

THE COURT: Thank you, Mr. Klaus. 

All right. And, again, we are still very much behind, so 
I'm going to ask, let's be as concise as we can. Who's going to 
argue on behalf of ASTM on ownership? 
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MR. FEE: Your Honor, I'm Kevin Fee from Morgan Lewis 
on behalf of ASTM and on behalf of all of the plaintiffs in the 
ASTM case. 

THE COURT: All right. Good morning. 

MR. FEE: Your Honor, Ms. McSherry started off the 
defendant's presentation by saying the core of this case has 
always been about whether or not incorporation by reference 
destroys the copyrights on standards written by private 
organizations, and we agree. 

Having said that, plaintiffs understand they have the 
burden of proving that they own the copyrights in this case, but 
the defendants have spent over three years trying to concoct 
arguments about why there are some holes in the ownership here. 

THE COURT: Well, let me ask you. Does the 
registration certificate for the 1999 Annual Book of Standards 
create the same rebuttable presumption of ownership for D39698 
and Dl217-93(98) as the registration certificates for those 
specific standards? And I single those two out because they're 
different from the others. Are those copyrighted individually? 
Is that in the record somewhere? 

MR. FEE: No, Your Honor. They're part of a 
compilation registration for the Book of Standards. 

THE COURT: Okay. 

MR. FEE: And, first of all, I want to note that the 
reason you're probably asking this question is we didn't have an 
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opportunity to address this in our briefing. It was raised in 
the final brief by Public.Resource. But anticipating that you 
might have that question, I have the answer here for you. 

The Book of Standards' collective registration covers all 
the individual works contained in that collection under a series 
of cases that have found that where an owner of a collective 
work also owns the copyright and the constituent parts of that 
collective work, that the registration for that collective work 
covers both the collective work and the constituent parts. 

Just a couple of citations for that. 

There's the Xoom v. Imageline case. That's 323 F.3d 279 
from the Fourth Circuit. There's also the Morris v. Business 
Concepts case, 259 F.3d 65. That's at page 68 for a pinpoint 
site, Second Circuit, 2001. 

THE COURT: All right. 

MR. FEE: So, because the Book of Standards were 
timely registered within five years of the first publication, 
then we are entitled to a presumption of ownership and validity 
with respect to those works as a result of that collective 
registration. 

THE COURT: All right. 

MR. FEE: So getting back to where I was a moment ago, 
we've gone through three years of litigation in this case now, 
and Public.Resource still has not been able to come forward with 
any evidence to rebut the presumption of ownership that we're 
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entitled to from those registrations. 

The simple fact is they have no evidence that anybody other 
than the plaintiffs in this case owns these works, and that's 
particularly important, I think. Your Honor, because there have 
been literally thousands of participants who have been involved 
in the creation of these works. And this litigation has been 
the subject of a lot of publicity in the standards-development 
community. 

And despite, I'm sure, the efforts by the defendants, 
everybody's awareness of these issues, not a single person in 
the thousands and thousands of participants who have ever been 
involved in the development of standards for these plaintiffs 
has been identified by the defendant as saying, you know what, 

I am the owner and exclusive owner of the copyrights of any of 
those works. 

And I think it's also important to note that it isn't good 
enough for them to poke a hole and then say, oh, you didn't get 
a perfect assignment from this one person out of the 10 people 
on this committee. 

They can't defend their infringement by saying the 
plaintiffs in this case only owned 80 percent of the copyright 
interest of the works in issue. They have to prove that 
plaintiffs owned literally no copyright interest in the 
standards at issue in order for them to have a defense based on 
ownership. 
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THE COURT: If I didn't find that you were entitled 
to the presumption on all the standards, have you sufficiently 
demonstrated a specific author of each of the six standards has 
assigned their ownership stake to you? 

MR. FEE: Well, Your Honor, there's a couple ways we 
have ownership other than the presumption that arises from this 
registration. First of all, we submitted evidence from all the 
plaintiffs in this case that their employees made contributions 
to these works. 

There's no dispute that if they made contributions in the 
course of their employment, then the plaintiffs in this case 
would own at least that copyright interest as a result of the 
work for hire doctrine, and as I pointed out before, as long 
as we own some ownership interest in the copyrights, that's 
sufficient for us to prevail in this claim. 

In addition, we have also provided evidence related to 
assignments as well. Maybe the most clear instance of that is 
the 2014 National Electrical Code. I believe even the 
defendants don't contest the validity of the ownership of the 
NFPA with respect to that code, because there's clear 
documentation that they agreed to be works for hire and that 
anything that wasn't a works for hire was assigned. 

But even with respect to the other works, I know, for 
example, with respect to ASTM, we identified specific language 
that were authored by employees of ASHRAE works for hire. And, 
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in addition, we do have assignments from some of the persons who 
were involved in the development of those works. 

In particular, I have the declarations of a couple of 
individuals, Mr. Jennings and Mr. Cummings, I believe his name 
is, who have identified their role in developing certain of 
these standards. 

They've clarified that they understood from the start that 
those standards were going to be owned exclusively by ASTM, and 
to the extent there was any complaint about documentation with 
respect to the assignments, we've confirmed and provided 
evidence that they did do the click-through assignments that are 
part of the ASTM renewal of memberships every year which 
provides that everybody understands that they have assigned all 
of their copyright interest in any of the works that they were 
involved in to ASTM. 

So, because Public.Resource cannot meet its burden of 
overcoming the presumption of ownership arising from the 
registrations, they do spend a fair amount of time trying to 
argue that they're not entitled to a presumption in the first 
place. They argue that because there was a mistake, supposedly, 
in the completion of the copyright registration forms that 
somehow the presumption goes away. 

But as we pointed out in our briefs, the overwhelming 
amount of case law stands for a proposition that even if there 
are mistakes in a registration, that does not affect the 
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plaintiffs' ability to bring the lawsuit or the presumption of 
validity and ownership that accompany that registration unless 
two factors are met. 

First, the mistake has to be material, and secondly, the 
mistake has to be made with the intent to defraud the copyright 
office. The defendants in this case cannot be either of those 
requirements. 

First of all, identifying the works as works made for hire 
was not a material mistake because it's undeniable that even if 
we had identified those works as joint works with us being one 
of the authors, that the copyright registration would have 
issued. So we cited a brief in our case on that point exactly 
where a court found that a work made for hire form from the 
registration was not materially impacted by the fact that it was 
really not a work made for hire, but the plaintiff still had an 
ownership interest in that work. 

And certainly there's no proof of an intent to defraud the 
copyright office. In fact, the only evidence with respect to 
intent on how these forms were filled out was the evidence that 
ASTM had contacted the copyright office to describe the 
circumstance and ask the copyright office for guidance as to how 
to complete these forms. And the copyright office told ASTM 
that the proper mechanism under these circumstances was to claim 
a work for hire, so there's neither a material mistake nor an 
intent to defraud the copyright office. 
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There is one case, I believe from the Third Circuit, that 
Public.Resource cites for a proposition that fraudulent intent 
is not required, but even that case does not stand for that 
proposition. 

The court sort of left open the question of whether intent 
in the Third Circuit alone is required to eliminate the 
presumption of validity and ownership, but it did not decide the 
issue, because it doesn't have to. All the other cases that 
have been cited, Your Honor, stand for the proposition that they 
both have to be material mistakes and made with the intent to 
defraud. 

So I think the easiest way to sort of support a factual 
finding of ownership here, as I mentioned, in addition to the 
presumption that arises from the registration, is the joint 
authorship point. A joint work is described or defined in the 
copyright statute as a work that is prepared by two or more 
authors with an intention that their contributions be merged 
into inseparable or interdependent parts of a unitary whole. 

In this case, there can be no dispute that all the 
participants in the standards development organizations 
understood that these works would be combined into a single 
standard at the end of the day, and Public.Resource does not 
argue otherwise. So, under the plain meaning of the language 
under § 101 of the Copyright Act, that's all that's required for 
a joint work. 
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Public.Resource does try to argue that any copyright or 
any contributions by the plaintiffs' employees in connection 
with this matter were not copyrightable, but they provide no 
evidence for that assertion. 

There's no description in their brief, for example, as to 
why the contributions that we've identified that were made by 
employees with respect to D975 are not protectable or 
copyrightable. They don't mention any of these standards at all 
in their briefs, and they have an obligation to overcome the 
presumption that those are not copyrightable. They just haven't 
even tried to do so. 

Now, Public.Resource also tries to get around the joint 
authorship issue by relying on Aalmuhammed , a Ninth Circuit 
case, for the proposition that joint authorship requires more 
than just an intent of all the authors to combine their 
contributions into a single unitary work, but it also requires 
an intention at the time of the creation that the parties 
understand that they will both jointly own the work. But that 
is certainly not the law in this circuit, and it is not the law 
according to the United States Supreme Court. 

In the CCNV case, the D.C. Circuit addressed a very similar 
issue where there is a dispute between two parties who were 
involved in the creation of a sculpture. Both parties, at some 
point in time, filed applications to register, so they certainly 
didn't have a joint understanding that this work was going to be 
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jointly owned at the time. 

The D.C. Circuit described those facts, if they remained to 
be the same after a remand, to be a textbook example of jointly 
authored works in which the joint authors co-owned the copyright, 
because one party basically did the sculpture of the person; the 
other party did the sculpture of a grate. Everybody knew they 
were going to be put together in a single unitary work, and that 
was all that was required for there to be joint authorship. 

Now, that case, of course, did go up to the United States 
Supreme Court as well, and the Supreme Court agreed with the 
D.C. Circuit's assessment of the parties' rights under those 
circumstances. It said that the parties would be joint owners 
if they prepared the work, intending that their contributions 
be merged into a separate or interdependent whole, and nothing 
else. There was no discussion about an intent requirement. 

Now, I know we're running very short on time, so I'm just 
going to deal very briefly with assignments. I'm sure when they 
get up, they're going to tell you somebody didn't sign a form or 
this language isn't appropriate for this particular form that 
they're going to show you. 

The problem that they have, among many, with respect to 
those arguments is they have the obligation, in light of their 
presumption of ownership, to show that every single participant 
who was involved in creating that work did not sign a form that 
assigned those works to the plaintiffs in this case. I don't 
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know what forms they're going to show you, but in their briefing 
they certainly have not linked any of the forms that they 
complained about to any particular works at issue in this case. 

For example, they haven't come forward and said, here are 
the authors of D975; let me show you the assignment forms for 
all those. None of those people signed the forms that were 
required to be signed in order to assign ownership. 

The bottom line is, with respect to the ownership, there 
are no magic words with respect to assignment. The intention of 
all the parties is clear. These plaintiffs have been publishing 
these works for over a century in some circumstances, always 
claiming to be the owner of the copyrights. Nobody has ever 
come forward and said otherwise. Public.Resource has no 
evidence of anybody ever claiming ownership, and as a result, 
they just can't meet their burden with respect to any complaints 
about assignment. 

But maybe even more importantly, they don't have the right 
to raise this argument. The courts have made it clear that you 
cannot defend your copyright infringement by saying, oh, I 
infringed a copyright, sure, but it's not the plaintiff's 
copyright; there's some defect in the assignment that entitles 
me to copy their works without any consequences. 

The courts have said that the point of the statute of 
frauds, a provision essentially of the Copyright Act that 
requires assigned writing, is to prevent disputes between 
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authors or claimed authors about who owns the rights in the 
works. That is not what we have here. Public.Resource does not 
claim to be an author in this case, and as a result doesn't have 
standing to raise this issue. 

Courts have — we've submitted a bunch of cases to 
Your Honor about this issue that have concluded as I've 
suggested here, but I think it also makes sense just to think 
for a second about what this would entail if we're going to do 
this and allow them to challenge assignments with respect to 
each of these works. 

Bear in mind, we have over 200 works in this case. 

Almost all, if not all, these works involve many, many authors. 
They would have, I suppose, us have a trial where for each work 
we say, okay, identify every one of the authors. There may be 
dozens. For each of those authors, what documents did they 
sign? For each of those documents that they signed, were they 
authorized by their employer to sign it? We will be here for 
years doing trials, and — 

THE COURT: No, we won't. 

(Laughter) 

MR. FEE: I think you got my point. 

THE COURT: I got your point. 

MR. FEE: So, unless you have any other questions, 

Your Honor, that's all I have. 

THE COURT: Thank you. 
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MR. HUDIS: Your Honor, Jonathan Hudis for the AERA 
plaintiffs. Hopefully, we'll make up some time here, because on 
ownership we have a very, very simple case. We have one work. 

Of the 16 joint committee members of the 1999 standards, 13 of 
them signed nunc pro tunc work made for hire agreements with the 
sponsoring organizations. 

The heirs of two deceased committee members signed 
posthumous copyright assignments. Those are all attached to 
Ms. Ernesto's declaration. To Register of Copyrights issued a 
copyright registration to these standards to AERA in 1999. An 
ownership of record was corrected by a supplementary copyright 
registration in the standards to all of the three sponsoring 
organizations in 2014. 

Public.Resource has not submitted any evidence to contest 
these facts of ownership, and in defendant's summary judgment 
brief, Public.Resource specifically elected not to move for 
summary judgment on this issue. 

So we have the registration certificates as prima facie 
evidence of validity and ownership, we have the work made for 
hire letters, the two assignments, all of which are of record; 
and as my colleagues from the ASTM case said, the assignee is 
not required to have been assigned a copyright by all of the 
co-owners to have standing to sue. We couldn't find one of the 
15. Just poof. He just could not be found. 

THE COURT: Mr. Hudis, I think that -- I have zero 
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minutes under the approximate schedule for arguments on 
Plaintiffs AERA, but if they're not contesting your ownership — 

MR. HUDIS: Well, let's hear from them. 

THE COURT: Right. What I want to do is I'll let you 
get back up if hear that they are contesting your ownership. 

MR. HUDIS: But like the ASTM plaintiffs said, they 
don't have standing to assert any problems with our copyright, 
even if they wanted to. Thank you. Your Honor. 

THE COURT: All right. Why don't we start with the 
standing issue. 

MR. BRIDGES: Thank you, Your Honor. The Supreme 
Court in Feist said the burden is on the plaintiff to prove 
ownership of a valid copyright and infringement of the 
constituent parts of a valid copyright. 

THE COURT: But isn't that in a case where there are 
disputed copyright holders? And what of plaintiffs' argument 
that you don't have standing to challenge their ownership of the 
copyrights in this case because you're not alleging that you own 
a competing copyright? 

MR. BRIDGES: Your Honor, the point is. Feist says 
the plaintiff has the burden of showing ownership in an 
infringement case. That was an infringement case. The Supreme 
Court said the plaintiff has the burden of proof of ownership. 

Now, they are relying upon a statement in the Copyright Act 
that says a registration within five years of first publication 
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is prima facie evidence. Doesn't say that a defendant doesn't 
have standing. It says it's simply prima facie evidence. 

And by the way, speaking about AERA, AERA is now relying on 
a 2014 registration, because it acknowledges that the 2009 
registration was wrong. So the 1999 registration was wrong. 

So it's not relying on the 1999 registration; it's relying on 
a 2014 registration. It's not within the five years. No 
presumption on error. 

But coming back to your point, the argument that they're 
basically making is that there's no standing to challenge 
standing. Standing is an Article III plaintiff burden. It has 
to show that it owns something. And, yes, it can have a prima 
facie case from the statute, but the statute doesn't say 
somebody accused of infringement can't challenge the first Feist 
factor. That's a red herring. 

There have been some cases that have said that, where I 
think they are cases where they're saying somebody's a dirty 
infringer; I'm going to throw the book at them. That seems to 
be the approach. It's almost like the fugitive disqualification 
doctrine or something like that. It doesn't play here. Feist 
made it clear that plaintiff has the burden. 

And in every copyright case brought by a U.S. author, 
there must be a registration. There must be a registration. 
Otherwise, you don't get into court. So the argument that a 
registration denies a defendant the ability to defend against 
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the first element of Feist makes no sense, Your Honor. 

Now I would like to go to the substance here because, 
frankly, yes, the ownership issues here are a dog's breakfast. 
Your Honor. They are a complete chaos, and I think it's — 

THE COURT: Why isn't it enough for the plaintiffs to 
demonstrate that they have at least one individual who will sign 
their authorship rights to the plaintiffs in each of the works 
at issue? 

MR. BRIDGES: That would be enough to give them 
standing, and we're not saying they don't have standing. But I 
would like to direct the Court's attention to a case involving 
one of the plaintiffs here, National Fire Protection Association. 

It had standing in its case when it was sued for copyright 
infringement by another code company. It had standing, it 
challenged ownership, and the district court. Northern District 
of Illinois in 2006, when the shoe was on the other foot, 
acknowledged that when NFPA was the defendant, it made some 
valid points about problems with the ownership. 

It said summary judgment would be inappropriate on 
ownership. It's clear that they don't own everything. There 
needs to be a trial to sort out what they do and don't own, 
because what they do and don't own makes a difference to what 
the alleged infringement is. So I absolutely ask the Court to 
read International Code Council v. National Fire Protection 
Association , 2006 Westlaw 850879, Northern District of Illinois, 
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2006. 

And what's interesting is that Public.Resource is just 
making the arguments here that National Fire Protection 
Association made there. Now it's changed its tune. But what's 
interesting is how many different ways the plaintiffs have 
changed their tune. If you read their briefs, they are all in 
on these being joint works. They're joint works. That's where 
they put all their force. 

Except that none of their registrations call them joint 
works. They didn't. And it's a material omission. Why? 

Because if a work is a joint work, all authors are to be named 
in the registration. All authors. And they didn't do that. 

And so the whole joint-works argument that you see now is just 
thrown up here. It wasn't in the registrations. It's thrown up 
here because they know they've got severe problems with the 
assignments. 

And I've given a copy of this to opposing counsel. I would 
like to hand this up. This, Your Honor, is a compilation of 
documents regarding ownership, and we have put a summary — I'm 
not asking the first one to be into evidence, but there's a 
summary on page 1 that you should consider part of our argument 
that explains the various, different types of documents. 

THE COURT: Is this in the record? 

MR. BRIDGES: Tabs 2 through the end are in the 
record. Your Honor, and they all have the filing stripes. 
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THE COURT: Tab 1 is the summary for — 

MR. BRIDGES: Tabs 2 through 27. 

THE COURT: All right. 

MR. BRIDGES: And, Your Honor, if you look at the 
summary in tab 1, every one of ASHRAE's supposed assignments are 
not assignments. They just aren't assignments. If you look at 
what is in tab 2, that's the document. 

It says, "I hereby grant ASHRAE the nonexclusive royalty 
rights, including nonexclusive rights in copyright." And down 
below, it says "nonexclusive royalty rights." 

A grant of nonexclusive rights does not convey an 
assignment. An assignment must convey exclusive rights of the 
copyright holder. There are no assignment documents from ASHRAE 
with any assignment language. It's all nonexclusive. So that's 
the first problem. 

The second problem is with ASTM. Bear in mind that the 
latest ASTM standard at issue is 2007, and it admits that it 
didn't ask for assignments until 2005. And then it later said, 
well, we sort of got assignments in our membership applications. 
But before 2008, they have no completed membership forms and 
therefore no assignments with the exception of one that really 
doesn't matter. 

It claims, well, we had an IP policy, but an IP policy is 
not an assignment. I mean, the copyright law is quite clear in 
§ 204. It says, a transfer of ownership is not valid unless — 
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I mean, it is not valid unless an instrument of conveyance or a 
note or memorandum of the transfer is in writing and signed by 
the owner of the rights conveyed or such owner's duly authorized 
agent. And the cases are clear that when you say on these 
membership forms, oh, I agree that anything I do will belong to 
you, that's not an assignment. So that's the ASTM problem. 

It's a severe problem. 

Then we get to NFPA, and I will admit that the most recent 
NFPA standard is better. Okay? It is absolutely better. 

That's why they amended the complaint to add it to the lawsuit, 
because it may be the only document at issue in this case where 
there looks like pretty good ownership. But even there, there's 
a problem, Your Honor, and this gets a little technical. 

Now that they claim that everything is joint works from 
joint owners, what about the fact that some of these joint 
owners are the U.S. government? That U.S. government employees 
participate as joint authors? 

No case has ever dealt with this. Your Honor, and I don't 
know how to deal with it. But § 105 of the Copyright Act says 
that U.S. government works are not subject to copyright, and 
Mr. Klaus explained that those are, where they're prepared by an 
employee acting in the scope of employment. Now they're saying 
they've got joint works with a whole bunch of federal employees 
as joint authors. 

So this is just a mess. Your Honor, yes. It is a dog's 


JA3358 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 101 of 142 
Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 337 of 441 101 


breakfast. It's a mess. Mr. Fee said that. They chose what 
case to bring. They chose how complicated to make it. They 
chose how vulnerable a set of standards they would choose. 

That's their problem. I think. Your Honor, there's no way they 
get summary judgment on ownership. 

I'm not necessarily saying that we deserve summary judgment 
on ownership, but the problem is this is a complete mess. It's 
a mess of their own creation, and it's a mess caused in part 
because they've changed their story as to what it is. Some of 
these things are nonexclusive licenses. Some they claim — they 
say in the registration, works made for hire. 

Well, there's a reason for that. Your Honor, because if 
it's a works made for hire, then people can't terminate 
assignments after 35 years the way they can if they're not works 
made for hire. There's a reason for that strategic point in 
copyright registrations. 

They claim, oh, we didn't mean anything wrong, because we 
were told by the copyright office. Your Honor, somebody reported 
what somebody said, something that happened years ago with no 
discussion about, well, what facts did they give the copyright 
office that caused the copyright office to say to do this? 

The problem is the whole thing is a mess. What we do know 
is that NFPA has been entirely hypocritical. We know that 
everybody has abandoned the very basis of ownership they claimed 
in their registrations that they don't want us to challenge. 
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It's just -- it's got to be done thoroughly. 

Unfortunately, ownership is on a work-by-work basis, and I 
notice that they brought this motion on only -- I think it's 
nine out of over 250 standards at issue. There's a reason for 
that. They've cherry-picked their best cases, and even then 
they've got a problem. 

And then one thing about joint authorship, they say, well, 
our staff were joint authors because we sort of helped add a 
footnote or we helped perfect some language or whatever. It's 
clear that in a law review — I don't want to say law review, 
because it's got its own structure, but if I submit an article 
to a law review and I own the copyright and the article, the 
editor at the law review who edits my law review article doesn't 
become my joint author. 

Having some editing function isn't an authorial function. 
And in many of these, the staff were forbidden from being 
members of the technical committees that actually did the 
writing, technical committees that had academics, government 
officials and the like. And Childress v. Taylor out of the 
Second Circuit makes it clear that an editor is not an author. 

I know we're running long, so I won't go any further. 

I would just say. Your Honor, there is no way that they've 
established ownership to the level that is necessary to get 
summary judgment for them on this. 

And I will say this. Now that they claim that it's joint 
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works, the Copyright Act — and remember, they claim they've got 
joint works, but they have not identified in any registration 
all the authors. It is important and it is material, because in 
the Copyright Act, it provides for the Court to consider 
bringing in the other owners. I'm not sure the other joint 
owners here know about this case, and if any one joint owner 
decides they like Public.Resource, that joint owner has full 
authority to grant Public.Resource a complete license. 

So they're saying, oh, we're joint owners with thousands 
of people. I think ASTM, across all its standards, says it has 
24,000 people. That's for thousands of standards, not just the 
standards here. But the point is, the Court has a responsibility 
to look to make sure the joint owners are protected, because if 
they are joint owners, they have a fiduciary duty to account 
their profits to the other joint owners, which is just another 
reason why it's such a specious argument. 

And why are they making a specious argument? Because what 
they said in the registration isn't right, and what they tried 
to do with the assignments couldn't turn the corner. So that 
was their third fallback, and it's intellectually dishonest. 

Your Honor, and should not be countenanced. Thank you. 

THE COURT: Thank you. 

MR. FEE: May I have one or two minutes, Your Honor? 
There was a lot in there. 

THE COURT: I'd prefer one, but I'll give you two. 
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MR. FEE: First of all, let's just cut to the chase 
with respect to the notion that it was somehow a material 
mistake not to list all the individual and joint authors. 

We cited a case, the Original Appalachian Artworks case, for the 
proposition that that's not a material mistake. The other side 
said nothing in their briefs. We've heard nothing about it 
today. 

The other notion that I want to correct for Your Honor 
is this notion that we are only claiming joint authorship. 

As we point out in the briefs, and as even the court in Veeck 
identified, organizations like this who are creating standards 
are the organizational authors of these works, but because they 
have literally no evidence to rebut the evidence we put in about 
what particular authors wrote while they were in our employment, 
that's the simplest way for you to dispel of this non-ownership 
issue. But we believe that we were the organizational authors, 
we have joint ownership at a minimum, and we also have 
assignments from the relevant persons. 

Again, we didn't see any evidence about assignments that 
were tied to any of the works in these issues. I don't think 
this book -- you know, I looked at whatever he pointed you to. 
You couldn't tell if that person ever made any contribution. 

That's also, I think, important with respect to the 
government point he's trying to inject here at the last minute. 
He's sort of hypothesizing about what contributions, if ever. 


JA3362 




1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


Case l:14-cv-00857-TSC Document 116 Filed 10/13/16 Page 105 of 142 
Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 341 of 441 105 


were made by federal government employees in the course of their 
employment. Then he's hypothesizing about a potential argument 
that that somehow affects the copyright interest here. There's 
no support for any of that in either the case law or in the 
record. 

I do want to turn just for one second to this ICC case, as 
well, that he likes to make a big deal about. The ICC case, 
first of all, there's two points that I think are important. 

One is the assignment issues in the ICC case were a little 
different than the ones that we have here in that there is also 
a provision that was not raised in the ICC case that is raised 
in this case as a basis for assignment. 

And similar language is also available to ASHRAE. If you 
look at the ASHRAE assignment that Mr. Bridges read to you — 

I think it was Exhibit 2. So he read one portion of that 
document to you. But in the section that has the No. 2 next to 
it, at sort of the end of that, it says, "I understand that I 
acquire no rights in publication of this standard in which my 
proposal in this or other similar analogous form is used." 

So there's a clear disavowal of any ownership right in 
these forms that was also present in the NFPA forms as well. 
That, combined with the fact that the NFPA has been claiming 
ownership for these works for over a century without any 
objection I think is more than adequate to show that there's an 
intent to assign, and this document suffices to meet the statute 
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of frauds requirement for the Copyright Act, assuming you even 
believe that they could raise that issue. 

Did Your Honor have anything else? 

THE COURT: No. Thank you. 

And I'll just say now that, given where we are with time, 
I'm not going to hear argument on the motion to strike the 
experts. I can rule on the papers on those unless you think 
there's something absolutely — and I apologize if somebody 
spent a lot of time preparing to argue that; but given where we 
are, I feel like the briefs have covered that, and I can rule on 
the papers on that one. 

Mr. Hudis, did you have something that your learned 
co-counsel didn't cover? 

MR. HUDIS: Only what Mr. Bridges just brought up. 

I'll take a minute. The '99 registration, yes. We are 
absolutely relying on that. The only thing that was changed 
from '99 to 2014 was to add the two other co-owners. A mere 
correction of ownership. We have cited the Billy-Bob v. Novelty 
case out of the Seventh Circuit, and it says they have no 
standing to challenge any of this. This was a mere correction 
of a mistake. It is not a material mistake, and anything that 
Mr. Bridges says otherwise is just not true. 

Merely providing comments, by the way, this is something 
that Mr. Bridges just said that was very surprising to me. 

Merely providing comments is not authorship. Well, then, we 
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have ownership and validity and authorship all wrapped up in a 
very nice, neat bow. There's no challenge on anything I heard 
from Mr. Bridges just now about the ownership of our copyright. 
Thank you. Your Honor. 

THE COURT: Thank you, Mr. Hudis. 

So on the trademark issue from ASTM? 

MR. FEE: Kevin Fee again. Your Honor. Just one more 
point, if you don't mind, on the copyright that Mr. Hudis just 
reminded me of. The evidence with respect to copyright 
ownership is not that there were just editorial changes made by 
the parties. We have declarations with respect to ASTM where 
we've identified entire paragraphs that were written by ASTM 
employees in the course of their employment. So the notion that 
we were adding a footnote or changing a comma here and there is 
just not consistent with the evidence. 

Now moving on to the trademark issues. Public.Resource, 
like its ownership story, has done its best to try to complicate 
this trademark case, which I think is really actually a 
relatively straightforward trademark case. 

Public.Resource has used exact copies of plaintiffs' marks 
on what it claims to be exact replicas of plaintiffs' standards, 
and it intends the public to believe that the materials that it 
posted on its website are authentic versions of the standards 
offered by the plaintiffs, when they simply are not. 

The fact of the matter is that the plaintiffs in this case 
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have absolutely nothing to do with the electronic files that 
Public.Resource posted on their website. Plaintiffs had never 
seen those files before they were posted on the Internet, and 
plaintiffs exercised no quality control over the files that the 
defendant posted on the Internet. And it certainly did not 
authorize Public.Resource to put those files — sure. 

The bottom line is, Public.Resource placed plaintiffs' 
trademarks and logos on knockoff publications that are of an 
inferior quality to the publications of the plaintiffs, and 
that is a clear-cut trademark infringement case for which 
summary judgment is warranted. 

Now, there is no argument here about whether or not 
plaintiffs own protectable trademarks. And when an identical 
trademark is used in connection with identical or very similar 
products,; it is not necessary for Your Honor to even walk 
through all the likelihood of confusion factors, and we cited 
numerous cases for that proposition in our briefs. 

And not surprisingly. Public.Resource doesn't cite a single 
case where a plaintiff failed to meet its burden with respect to 
trademark infringement when there is evidence of the exact same 
mark being used in connection with very similar services when 
there is an intent to have consumers believe that the source or 
origin of the defendant's product was the plaintiff. 

THE COURT: Let me ask you. You argue that the 
defendant's double-keying method is not as effective as the 
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triple-keying method for guarding against inaccuracies, but as 
I understand the doctrine, I should be able to look at evidence 
of your quality control standards to determine that defendant 
hasn't met them. 

Did you put in any evidence of your own quality control 
standards, and if so, where is it in the record? 

MR. FEE: I believe that if you look in the ASTM one 
I'm most familiar with, Mr. Tom O'Brien's declaration, there is 
a description of those quality control methods. 

THE COURT: All right. 

MR. FEE: But I would point out that I think 
the bottom line is that that doesn't really matter in this 
circumstance except with respect to harm, which you may hear 
about later. 

THE COURT: Right. 

MR. FEE: But whether or not — you know, they could 
have done a perfect job complying with our quality control 
standards. They still don't have the right to steal our 
trademarks and put it on something we have nothing to do with. 

Because there's no real good argument for the assertion 
that you could use the exact same mark on virtually identical 
products without avoiding infringement, the primary argument 
that we hear from Public.Resource is that you can't bring a 
trademark case in this circumstance, and that argument is based 
entirely on the Supreme Court's decision in Dastar . 
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But the very first sentence of the Dastar opinion starts 
with Justice Scalia saying that the issue before it was "whether 
§ 43(a) of the Lanham Act prevents the unaccredited copying of a 
work." 

That is not the issue in this case. In fact, it's the 
exact opposite. This is not an unaccredited copying of a work. 
It is placing a party's trademark on a work that the plaintiff 
had no involvement in the product that bears its trademark. 

But the Supreme Court in that case decided that it must assess 
whether or not § 43(a)(1)(A) of the Lanham Act's use of the term 
"origin of goods" covered just the person who made the physical 
good or whether it was the person who created the expression. 

The Court in that case held that "origin of goods," as that 
term is used in § 43 (a), covers just the physical good at issue 
and not the person who created the expression that might be 
embodied in that good, and it reached that conclusion because 
it wanted to avoid the possibility of there being a perpetual 
copyright for the expression after the copyright had expired or 
otherwise gone away. 

So the Court noted that "The rights of a patentee or a 
copyright holder are part of a carefully crafted bargain under 
which, once the copyright monopoly has expired, the public may 
use the invention or work at will but" — and this is 
important — "without attribution." That is not what happened 
here. 
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On the other hand, the Supreme Court noted, "A party could 
face Lanham Act liability for crediting the creator if that 
should be regarded as implying the creator's sponsorship or 
approval of the copy." And that's exactly what's happened here. 

So Dastar actually confirms that a trademark infringement 
case is possible in this circumstance, not the opposite. 

But you don't have to take my interpretation of Dastar . 
We've cited many cases that confirm this is how Dastar 's 
properly interpreted. In the Bock case, the Court held that 
Dastar stood for the proposition "that the origin of goods 
provision in 43 (a) of the Lanham Act does not contain a cause of 
action for plagiarism." That's true. If we were complaining 
about the unattributed copying of our text, then Dastar would 
bar that claim, assuming we didn't have a copyright infringement 
claim. 

On the other hand, the Slep-Tone case that we cited 
indicated that Dastar suggested that "there would have been a 
Lanham Act violation where, for example, Dastar had simply 
copied the television series and sold it as Crusade in Europe 
without changing the title or packaging, including the original 
credits to Fox. 

So just like in our case where they don't change the 
original crediting to the plaintiffs, the Slep-Tone case 
concluded that a trademark case could be brought in conjunction 
with a copyright infringement case in that circumstance. 
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Public.Resource really only cites one other case in support 
of its argument; but that case also involved an attempt to 
convert a plagiarism case into a trademark infringement claim, 
and that was the Prunte v. Universal Music Group case. 

Public.Resource also has tried to defend its conduct under 
the first sale doctrine, but the first sale doctrine applies 
only to goods that are being sold when those goods are the 
genuine product of the plaintiff that are being resold to 
consumers. The electronic files that are being sold by the 
defendant in this case were posted, are not the authorized 
documents that were created by the plaintiffs, and therefore 
are not subject to the first sale doctrine. 

Public.Resource had purchased hard-copy materials from 
the plaintiffs, and if they had wanted to repackage those or 
do something with the hard copy that they had, that would be 
covered by the first sale doctrine. But that's not what they've 
done here. 

Instead, they've created new documents or electronic files 
of what they purchased from the plaintiffs and tried to defend 
that under the first sale doctrine, but the bottom line is that 
those electronic files were never purchased from the plaintiffs 
in this case. Making things worse, of course, they're of a 
lesser quality than the plaintiffs' works. 

Defendants also try to defend their use of the plaintiffs' 
trademarks by claiming nominative fair use, but there's three 
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requirements that prove nominative fair use. One is that the 
use of the plaintiffs' mark is necessary to describe the 
plaintiff's product. But there's no reason that the plaintiffs 
need to refer to ASTM if what they're really trying to publish 
is the law. They could just publish what they call "the law" 
without reference to ASTM or the other plaintiffs, and that's 
exactly what happened in the Veeck case. In Veeck , the Fifth 
Circuit noted that Veeck had just identified the building codes 
as the law as to relevant towns and not as the model codes 
themselves, which is what is being done here. 

The second requirement for non-fair use is that the 
defendant only use as much of the plaintiffs' trademark as is 
necessary. It's not necessary, as I just explained, for them to 
use any of our marks, but it certainly is not necessary for them 
to use the logos of our clients. 

There's a long line of cases that we've identified in our 
brief that stand for the proposition that it's very unusual, if 
not almost never the case, that you have to actually use a logo 
as part of a nominative fair use. If they had to use our name 
at all, they could just call it ASTM Standard D975. They don't 
need our circle and our symbol there. There's no way to argue 
otherwise. 

It's even pointed out and made more clear by the fact that 
Public.Resource, after the fact now, has started to post some 
standards not at issue in this case, but other standards of 
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plaintiffs where they don't put the logo on there. So they 
obviously don't need to have the logo there. 

The third requirement for nominative fair use is that the 
defendant not do anything that suggests sponsorship or 
endorsement by the plaintiffs of the works that are being 
provided by the defendant. But Public.Resource, the testimony 
is clear, did everything in its power to try to make the 
standards that he was posting or that Public.Resource was posting 
on the website to look exactly like our standards. So there's 
no basis for the notion that they did anything to avoid a 
likelihood of confusion in their supposed nominative fair use. 

The last point I want to touch on real quickly is the 
notion that some disclaimer is present and that somehow that 
will eliminate the likelihood of confusion. 

First of all, it bears noting that the defendant has the 
burden of proof with respect to showing that a disclaimer will 
eliminate the likelihood of confusion. The CFE Racing case, 

793 F.3d 571, from the Sixth Circuit so holds, as does Weight 
Watchers v. Luigino's , 423 F.3d 137. 

Public.Resource presented literally no evidence that any 
disclaimer would be effective in this case. In fact, the truth 
of the matter is, with respect to the standards at issue in this 
case, there are no disclaimers at all. 

You saw the sort of cover sheet you were referring to 
earlier with the red, white, and blue stripes on there which I 
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think Public.Resource likes to suggest is a disclaimer of some 
sort, but that disclaimer says nothing about not being affiliated 
with the plaintiffs in this case or that Public.Resource has 
authored these materials in any way. After the fact. 

Public.Resource submitted some evidence of a disclaimer, but it 
has nothing to do with any of the works in connection with this 
matter. In any event, a proper disclaimer is not sufficient in 
this case. 

As the court in the International Kennel Club case in 
the Seventh Circuit recognized, quote, "especially where 
infringement in the case is verbatim copying of plaintiff's 
name, we are convinced that plaintiff's representation and 
goodwill should not be rendered forever dependent on the 
effectiveness of fine-print disclaimers often ignored by 
consumers." 

The thing that's most prominent and that tells the 
consumers in the first instance who is the source of these 
materials are the logos of the plaintiffs in this case. 

That's what parties are going to look at when they're trying to 
figure out who was responsible for these files. If you have some 
sort of disclaimer on it, it's going to be ignored. That's why 
courts frequently don't find disclaimers to be sufficient to 
avoid confusion. 

Unless Your Honor has any other questions, that's all I 

have. 
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THE COURT: Thank you. 

MR. BRIDGES: Thank you, Your Honor. This is 
Andrew Bridges again for Public.Resource. 

THE COURT: Mr. Bridges, if the defendant's sole 
purpose is to disseminate the law, as you say, why do you need 
to disseminate the plaintiffs' logos? 

MR. BRIDGES: We don't have to. Your Honor, except 
that what we've done is, in the spirit of what we understand the 
incorporation is to be, which is incorporation of particular 
documents, Public.Resource has replicated the entire document. 

As is. Now, we need — 

THE COURT: Well, then you add this certificate; right? 

MR. BRIDGES: That's right, which emphatically 
makes the point that it is the law. It doesn't say this is 
Public.Resource's. We need to be clear. The allegations that 
Public.Resource is trying to confuse the public about source 
sponsorship or affiliation of these standards is pretextual and 
ironic. The fact is, they would sue Public.Resource no matter 
what. If Public.Resource dropped the logos, they would sue for 
reverse passing-off, but because it maintained the logos, 
they're suing for trademark infringement. 

Let me be clear. Public.Resource would take direction from 
this Court. Logos: yes or no? It doesn't care. It simply 
tried to replicate the law which consists of these documents 
incorporated by reference. 
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Disclaimer. First of all, the Supreme Court in two cases 
has approved disclaimers. If Public.Resource needs to say — 
first of all, I'm not sure that the plaintiffs would want their 
logos taken off because they use their monopoly position to try 
to make money by associating these standards that have become 
law with themselves. But if they want the logos off, we will 
get the logos off. Your Honor. That's not a sticking point. 
We're just trying to make clear that these are the laws that are 
in the CFR or state law or whatever. If the Court wants a 
disclaimer — 

THE COURT: Well, with regard to disclaimer, if you 
point to your disclaimers as sufficient to notify consumers that 
the standards aren't originals, that they're reproductions, I 
look at the language on the cover page, and it's hard to 
understand how this — is this Exhibit 16? — how this resolves 
any confusion. 

MR. BRIDGES: Your Honor, it's not just about this. 

It's about the entire experience that somebody has going to 
Public.Resource's website. When I go to the Cornell website, I 
don't think I'm going to the Library of Congress to get a law. 

I know I'm going someplace where I can get the law. I've got no 
confusion between the National Archives and Cornell, but I know 
that I can go to Cornell to get the law. There is no likelihood 
of confusion that somebody thinks Public.Resource wrote these. 

THE COURT: Then why do you have a disclaimer? 
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MR. BRIDGES: We have this document that says this is 
the law. We have — and I'm not — there are different 
disclaimers at different times, so I'm not clear on exactly what 
they've all been. 

THE COURT: Why do you even need this? 

MR. BRIDGES: We need this to make a political point 
that this is the law, and we want people to understand that this 
is no longer just somebody's private standard. This is the law, 
and that's exactly what it says here. It's giving the citation 
to the U.S. Code that makes it the law. 

THE COURT: If all you want to do is to make sure that 
consumers realize that it is the law, why do you need their logo? 

MR. BRIDGES: I'm saying. Your Honor, we would drop 
the logo in a second if that's the Court's direction. The 
reason we included the logo — we don't have to have a fight 
over them with this. 

THE COURT: Well, they brought a claim. 

MR. BRIDGES: That's right. They brought a claim, and 
they would have brought a claim no matter what we did, because 
it's really a copyright issue. 

THE COURT: The Court is unconcerned with their 
motivations for bringing a claim. My only concern is whether 
they have a valid claim. 

MR. BRIDGES: Your Honor, if the motivation is to 
enforce a copyright right, then it's squarely in the middle of 
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Dastar , and that's a problem. That's why the motivation is 
relevant. If it is to get around a limitation imposed by the 
Copyright Act, then it's a Dastar problem. 

But let me make it clear. We're trying — we don't — what 
we want is to continue to make the law available. It doesn't 
matter if it is with a logo or without a logo. We just want to 
make the law available. But they would have sued us for 
dropping the logo as well as for including the logo because they 
don't want the standards out there. And that's the copyright 
issue. This is really a copyright case. 

So if the Court says drop the logos, they would be dropped. 
If the Court says add a disclaimer that says you have scanned 
and reformatted these, we would add that disclaimer. If you 
want to say Public.Resource had no involvement in the creation 
of these standards, that's fine. Public.Resource has no desire 
to create any confusion. 

As a matter of fact. Public.Resource tries to be very clear 
about what these are. If anything, the plaintiffs want everybody 
to think you have to buy the law from them, and that's the 
problem in this case because they're saying they've got an 
exclusive right to the law and they have the right to control 
who accesses the law, who makes a derivative work of the law and 
so forth. 

So this trademark issue need not be an issue, because 
Public.Resource isn't trying to make a point about itself other 
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than to be clear about what it's doing. So there is — we can 
fight the trademark fight, but we don't need to fight a 
trademark fight. Your Honor. 

THE COURT: All right. Thank you, Mr. Bridges. 

MR. BRIDGES: Thank you. 

THE COURT: Any discussion of remedies? 

Good afternoon now. 

MR. CUNNINGHAM: Good afternoon. Your Honor. 

Blake Cunningham of King & Spalding. I represent Plaintiff 
ASHRAE. I'll be speaking on behalf of the ASTM plaintiffs on 
this topic. I'm mindful of the time, so I'll try to keep this 
very brief. 

Now, Your Honor, the Supreme Court counseled, in the 
eBay v. MercExchange case, that there are four essential factors 
that should be considered when a court is deciding whether to 
exercise its discretion to issue a permanent injunction. 

The first of these factors is whether or not irreparable 
injury will occur in the absence of an injunction. Now, here 
it's not disputed that plaintiffs' standards have been accessed 
thousands of times on defendant's website. It's also not 
disputed that defendant placed plaintiffs' standards on the 
Internet Archive website and that they were downloaded thousands 
of times from that site. 

That these downloads and accesses would represent some 
impact on the legitimate market for these works is, as Your 
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Honor noted earlier today, somewhat a matter of common sense. 

But in this case, we've also backed it up with the expert 
opinion of Mr. Jarosz, which of course went unrebutted. 

THE COURT: Let me ask you — and I don't mean to jump 
around, but while I have you up here. You've moved to summary 
judgment as to six standards. At this time, are you still 
seeking a permanent injunction just as to those six? 

MR. CUNNINGHAM: So we are seeking a permanent 
injunction — I think it was nine standards. Your Honor, that we 
moved on. So we're seeking a permanent injunction for those 
nine standards. We're also asking that the Court enjoin future 
infringement. We've cited a number of cases in our briefs where 
courts have enjoined future infringement of separate works, and 
here we think that's especially on topic because Public.Resource 
-- I think even earlier today Mr. Bridges stated that they plan 
to keep posting more and more works, and it would not be 
efficient for any of us if we have to keep coming back and 
reliving this same case. 

THE COURT: You seek to cure the copyright 
infringement broad enough to cure any trademark infringement, 
as well as — from what I hear, everybody's willing to be 
reasonable on this, but — 

MR. CUNNINGHAM: Yeah. I think an injunction on the 
copyright infringement would tend to also encompass the 
trademark issues. 
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THE COURT: And what's your intention regarding your 
remaining contributory copyright infringement claim? 

MR. CUNNINGHAM: If we got an injunctive relief that 
involves taking the standards off the website, I don't think we 
would intend to keep pressing for any sort of damages or 
anything on a contributory theory. 

THE COURT: All right. 

MR. CUNNINGHAM: So, Your Honor, the kind of question 
becomes, when looking at the harm here, whether the harm is 
itself irreparable. Now, courts have looked at this question of 
what makes harm reparable or irreparable, and the Second Circuit 
in the Salinger v, Colting case took up this question and said 
the following: 

"Harm might be irremediable or irreparable, for many 
reasons, including that a loss is difficult to replace or 
difficult to measure, or that it is a loss that one should not 
be expected to suffer." 

Now, in this case, I feel like there are at least three 
reasons why the harm that's suffered would be very difficult to 
measure and difficult to compensate with monetary damages. 

The first is, as our expert Mr. Jarosz went into detail on, 
one of the likely outcomes of this case is that plaintiffs would 
have to change their business models. If we lose the revenue 
from selling standards, we may have to switch, for instance, to 
a business model where we charge people to participate in the 
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standards creation process. 

Now, our clients feel like that would result in less 
preferable standards that don't reflect the broad interest that 
we currently try to reflect in our standards creation. They may 
also be the result that we would produce less standards, fewer 
standards. Again, that's a negative outcome for us, but it's 
one that's particularly hard to quantify. 

A second reason why damages might be hard to quantify here 
is that the works are shared without restriction online by the 
defendant. This leads to an outright loss of control by 
plaintiffs of their copyrights. The works can be downloaded, 
printed, and even redistributed by anyone. And Public.Resource 
notably does not have information on how the works are used 
after they're downloaded, which means that we can't even know 
the full extent of the infringement here. 

Now, this is very much analogous to the 2007 Grokster 
case which we discuss in the briefing. In that case, the 
defendant was being sued for marketing a peer-to-peer 
file-sharing network that facilitated widespread sharing of 
files, and the Court found irreparable harm because the nature 
of the defendant's conduct and the redistributable nature of the 
works rendered the works "particularly vulnerable to continuing 
infringement on an enormous scale." 

The Court went on there to say, "When digital works 
are distributed via the Internet, every downloader who receives 
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one of the copyrighted works is, in turn, capable of also 
transmitting perfect copies of the works. Accordingly, the 
process is potentially exponential rather than linear, 
threatening virtually unstoppable infringement of the copyright." 

And we feel like we're in the same situation here. 

Defendant has shared our works without restriction, we have no 
view into how they're being used down the line, and there's 
virtually unlimited infringement happening. So it represents an 
outright loss of control of our copyrighted works. 

The third thing I wanted to get into in terms of why harm 
would be incredibly difficult to quantify here is that there's 
reputational harm. It's not disputed, I think, that our 
clients, the plaintiffs, have spent decades, if not over a 
century, building their reputations by producing quality 
standards. And if these are recreated in ways that include 
errors, include substantive errors, then that could be 
potentially damaging to the reputation of our clients. 

And as Mr. Fee explained in his argument, this is not 
necessarily a purely theoretical argument. We do believe that 
Public.Resource's quality control mechanisms have been quite lax 
and have resulted in some substantive errors. One that I'll 
provide as an example, in Mr. Pauley's declaration, Mr. Pauley 
from NFPA described how Public.Resource's OCR process had changed 
the letter M, which stands for meters, into two letters, I and 
N, which of course could be an abbreviation for inches. 
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So it's not hard to see that these errors could lead to 
real substantive changes in the works, and we feel like our 
clients should not be forced to suffer the kind of reputational 
damages that come along with these type of errors. And in fact, 
the law is pretty clear on this. We cited two recent cases from 
this circuit, the Breaking the Chain case and the Hanley-Wood 
case that said that where there's a continued threat of 
infringement that could harm the reputational interest, that 
that in fact does justify an injunction. 

Now, defendant, for its part, they can't come up here and 
tell you that absolutely there is no harm that exists. Instead, 
they're going to try to shift the dialogue here to say that 
there's not very much harm or enough harm. They're essentially 
trying to import a fifth factor into the eBay test and say that 
there must be a severe harm. But that's not really the standard 
here, Your Honor. The standard is whether the harm at issue is 
irreparable, and the bar is much lower than defendants would 
suggest. I'll refer again to the Grokster case. 

In that case, the court stated, "Irreparable harm may not 
be presumed, but in run-of-the-mill copyright litigation, such 
proof should not be difficult to establish." And then the court 
went on to explain that loss of market share and reputational 
harm were prime examples of how that could be established. 

Similarly, the Second Circuit in Salinger v. Colting 
speculated that, even after eBayj fr ;as an empirical matter, most 
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copyright cases would likely involve some form of irreparable 
harm. And then the court went on to say, "The historical 
tendency to issue preliminary injunctions readily in copyright 
cases may reflect just that." Put simply, the burden is not so 
high as the defendant suggests when it comes to irreparable 
harm. 

The second one of the factors which I'd like to discuss 
quickly is whether or not there are other remedies available 
that would be sufficient here. As I've already explained, it 
would be very hard to quantify what damages would be in this 
situation, but even if you could do so, I think it's not 
necessarily contested that defendant has no willingness or 
ability to pay damages here. 

In fact, if you look at the briefing, the defendants were 
silent on this one of the four gBa|r factors. They essentially 
conceded it, and there's a reason for that. We've got 257 works 
at issue in just the ASTM case. Statutory damages in the 
copyright scenario can be up to $150,000 per work for the kind 
of willful infringement that we've got here. So you're looking 
at tens of millions of dollars in potential damages and a 
defendant who has very, very limited resources and no ability to 
pay that. So there are no monetary damages really available 
here, and that's why we've chosen to bring this case and ask for 
an injunction. 

Now, one other thing I'd like to say on that is, because 
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the monetary relief here is really inadequate, if the Court 
finds for us on the merits, the only prudent thing to do would 
be to issue an injunction. We can't be in a situation where 
it's kind of winner takes nothing, where we don't get an 
injunction or damages, and the damages here aren't available. 

So if Your Honor did find for us on the merits but didn't 
find that injunction was warranted, I guess our only option 
would be to, next time Mr. Malamud posts a standard, actually to 
sue him again and this time to ask for damages. I don't think 
that that would be an efficient outcome for the defendants or 
the plaintiffs or the court. 

The third factor. Your Honor, to consider under eBay is a 
balance of the hardships. This is a particularly easy factor 
here because we have deposition testimony from Mr. Malamud where 
he essentially admits that there would be no harm to 
Public.Resource. Mr. Malamud was asked at his deposition: 

"If Public.Resource was unable to continue to post the 
standards incorporated by reference on its website, what impact, 
if any, would that have on Public.Resource's financial ability 
to survive long-term?" 

He stated, "Probably none." 

Mr. Malamud was also asked if he could identify any way in 
which Public.Resource would be harmed. The only thing he could 
think of was that there might be potential wasted effort in 
posting these standards online. But, of course, this wasted 
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effort is legally irrelevant since an infringer cannot claim an 
equitable interest in its infringing conduct. I would direct 
the Court to the Fox television case for that proposition. 

Now, the final of the four factors that I’d like to talk 
about is the public interest. This has already been covered to 
some degree in the earlier arguments today, so I won't go into 
the details other than to say that there is a public interest in 
promoting the creation of creative works. 

In this instance, we feel that's especially important since 
the works here serve the public good. Even Mr. Malamud has 
admitted that these are important works. He's stated that 
NFPA's works, quote, "save lives." And we've got the opinions 
of Mr. Jarosz and in amicus briefs where we've seen that if an 
injunction doesn't issue here, there's a real fear that the 
quantity and quality of these works will be diminished. 

Now, we have to balance that against the public interest 
that Public.Resource claims that it serves, which is increased 
access. But I think as we've heard a lot about earlier today, 
there is really no access issue here. Mr. Malamud is kind of 
the lone complaining voice when it comes to access to these 
standards. There's no evidence that anyone who really needed to 
use these standards has not been afforded access, and we already 
provide access in our reading rooms. 

So when you balance these two things, I think it's pretty 
clear that this factor, as well as the other three factors that 
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we've discussed, weighs in favor of granting an injunction. 

Thank you, Your Honor. 

THE COURT: Thank you. 

MR. HUDIS: Good afternoon, Your Honor. Jonathan 
Hudis for the AERA plaintiffs. This is on the right to relief. 

As Mr. Cunningham cited the eBay four factors for 
entitlement to a permanent injunction, I won't reiterate them 
for the Court now but just to go through the factors as unique 
to our plaintiffs in the 14-857 case. 

As the sponsoring organizations have established the threat 
of Public.Recource's continuing infringement, they're entitled 
to an injunction. That's the Green v. Brown case in this Court, 
DDC 2015. Public.Resource's stated goal and mission is to 
publicly post standards incorporated by reference into federal 
and state law. Public.Resource still has an unauthorized 
copy of the sponsoring organization's standards on its server, 
as does the Internet Archive. 

It would be very simple for Public.Resource to repost the 
1999 standards to Public.Resource's website and to the Internet 
Archive with little effort. Mr. Malamud further admits that he 
will strongly consider posting the 2014 standards to the 
Internet if they are incorporated by reference to law, and that 
was repeated by Mr. Bridges here today. 

Thus, absent the issuance of a permanent injunction. 

Public.Resource will continue to disseminate plaintiffs' 
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standards without authorization. 

To the factor of irreparable harm, the Court should 
properly look to the future threat of injury to the sponsoring 
organizations. Number one is plaintiff's inability to prevent 
further viral infringement, and we cited, among many cases in 
our briefing, the Walt Disney and Hanley-Wood cases in this 
circuit. 

The damage has already been done with respect to the '99 
standards that were published for the two years online. The 
2014 standards were announced in 2011, at which point there was 
a 27 percent drop in the sales of the 1999 standards. Then in 
2012, the year that Public.Resources posted the infringing 
copies of the 1999 standards to the Internet, there was a 
further 34 percent drop in sales, and then the sales stayed 
suppressed in 2013. 

The 1999 standards are used in many graduate courses. 

The sales to students should have remained constant year after 
year until the release of the 2014 standards in August of 2014, 
and that was testified to by Professor Geisinger, both in his 
declaration and in his deposition. 

So again, same with the ASTM plaintiffs. The sponsoring 
organization's inability to measure sales losses due to 
Public.Resource's acts of infringement and contributory 
infringement, the funds which otherwise would be used for saving 
up to underwrite the cost of developing future updated standards 
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would be in jeopardy. 

There would also be two — excuse me — three adverse 
effects on the quantity and quality of the effort the joint 
committee selected by the sponsoring organizations put in to 
creating and updating the standards. If their work can be 
freely distributed on the Internet immediately upon publication 
and incorporation by reference — 

THE COURT: Slow down a little bit, Mr. Hudis. 

MR. HUDIS: Slowing down — potential future joint 
committee members and the sponsoring organizations themselves 
will lose incentives to update this work. 

Finally, as to irreparable harm, would be the inability to 
inform the public that the 1999 standards are no longer the 
latest version, and the public should purchase the 2014 version 
instead. This harm to the public would be highly damaging to 
the sponsoring organizations' collective reputations. 

The balance of hardships. In contrast to the significant 
harms to the sponsoring organizations if a permanent injunction 
is not granted, Public.Resource has no cognizable interest in 
continuing to infringe our standards and our copyright. 

As an infringer. Public.Resource cannot complain about its 
loss of copyright to offering an infringing substitute online, 
and that's the WPIX case we cite in our briefs. It therefore 
will suffer no recognizable harm if a permanent injunction is 
entered. 
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Finally, the public interest, Your Honor. Here the public 
interest favors entry of an injunction to stop further copyright 
infringement. The object of copyright law is to promote the 
store of knowledge available to the public. The Copyright Act 
accomplishes this by providing a financial incentive to 
contribute to the store of knowledge. 

Allowing Public.Resource and others to freely copy the 
sponsoring organization's standards will detract from the 
important store of knowledge, recommended best practices for 
testing, design, and administration available to the public. 

If plaintiffs do not have continuing incentives to secure 
copyright protection, those incentives to have updated standards 
in the future will be lost. 

Unless Your Honor has any questions, those are my remarks. 

THE COURT: Thank you. 

MR. HUDIS: Thank you, Your Honor. 

MR. BRIDGES: Your Honor, while I think we agree that 
eBay lias stated the facts, one thing eBay also said was success 
on the merits alone does not justify an injunction. So I think 
that much is clear. I want to move quickly through the first 
three factors and focus a bit on the fourth factor. 

The question as to whether plaintiff has suffered 
irreparable injury. So ASTM's president conceded, in an 
internal document, "To date, all of Public.Resource's postings 
have not had a measurable effect on our finances." 
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So they have relied upon two experts. I'll let the motions 
to strike speak for themselves, but they are extremely weak, and 
that's trying to be very charitable. They are not competent 
evidence. There are no qualifications that are appropriate for 
them. It's just serving as mouthpieces for things that 
witnesses should have been saying on their own and 
cross-examined on, and their methodologies were appalling. And 
that's what they needed to show actual harm. 

I want to go back to this point in Mr. Geisinger's report. 
He completely whiffed on the — 

THE COURT: What would be an appropriate remedy? If I 
found for the plaintiffs, what would be an appropriate remedy in 
your case if there is no irreparable injury for an injunction? 

I assume you're not going to say, oh, we are able to pay money 
damages. What would be the appropriate remedy? 

MR. BRIDGES: I am not able to say. Your Honor, 
because we feel that the public interest here is huge, and I'll 
have to address that. If the Court decides the Court is 
inclined to grant an injunction, then I would suggest that we 
have a separate round to address details. But it's just not 
appropriate here, for a variety of reasons. 

In HathiTrust , the court — well, that's in hardships. 

I'll get to that later. But the experts here were their 
substitute for facts, and their experts did not provide valid 
bases for claiming irreparable harm. 
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What's interesting is, they sort of concede that, because 
they move their focus to, well, we've lost control. We've lost 
control. Well, that's like saying our copyright's been infringed 
because that's what it means to have a copyright infringed. 

So they're sort of falling back on what eBay says is not 
important, not relevant, which is mere success. 

Then they said, oh, but we would suffer reputational harm 
because people will mistake our product. Well, that is very 
fixable, and that's not irreparable at all. I guess it could be 
repaired with a very, very modest injunction which says, put in 
a disclaimer and say the standards organizations are not 
responsible for this transcription. But they worry about that. 

And it's very curious that they mentioned, oh, the problem 
of quality standards. There's a reference to Mr. Pauley's 
declaration, and it's really instructive. Your Honor, because 
Mr. Pauley highlighted a dangerous error. He said in paragraph 
54, one passage left out the phrase "cables rated above 2,000 
volts shall be shielded." That was a major mistake, he said. 

It was NFPA's mistake. It was an error that NFPA corrected 
with an erratum. Why did Public.Resource omit it? Because the 
law of incorporation by reference is very clear. Incorporation 
by reference applies only to the specific document, and it does 
not extend to any corrections or revisions. 

So, in fact, this was not an error on Public.Resource's 
part; it was an error on ASTM's part. But because 
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Public.Resource is putting out there the very document that is 
incorporated by reference, it was accurate. NFPA's inaccuracy 
became the law. And maybe that's important for people to know 
about, and if so, that's something that Public.Resource shows 
people: This is what got incorporated, and if it's a mistake 
that NFPA had to correct, well, then as an incorporated law, 
it's missing something important. So this is very, very key, 
and this is actually a reason why Public.Resource's work is good 
and important, because it's telling people what the law is even 
when NFPA wants to recast what the law really is. 

The question of remedies at law are inadequate to 
compensate for the injury? Well, the presumption is first there 
has to be a showing of actual injury, and there just hasn't been. 
There's a nullity to consider whether remedies are inadequate to 
compensate for the injury when they haven't shown injury, and 
they like to retreat behind the thing, oh, the damages are 
unquantifiable. 

Well, that's what expert — competent experts would usually 
do, and they didn't have competent experts here. And we have 
again the admission from ASTM's president, no measurable effect. 
The plaintiffs' experts didn't analyze what happened in Veeck . 

They're saying here that there would be terrible harm if 
they suffer what actually happened in Veeck , and nobody showed 
that the standards development organizations had to go out of 
business or couldn't afford to do standards anymore because 
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Veeck said they had no right to monopolize them. There was 
a case study that their experts chose not to consider. The 
methodology just makes my mind explode. So they just don't have 
evidence on this. 

Let's go to the balance of hardships, because this is 
important. Again, it assumes actual injury. The Second Circuit 
said, when it was discussing hardship for a different purpose — 
it was a standing question. But the Second Circuit said, 

"The mere possibility of a future injury, unless it is the cause 
of some present detriment, does not constitute hardship." 

So what is the hardship, they say? Well, the hardship is, 
Your Honor, we've had a business model for a hundred years, and 
it would be hard for us to change it. Well, antiquity is not a 
virtue, and antiquity doesn't deserve for its own sake — the 
fact that this business model has been here a hundred years 
doesn't mean that that's what the business model should always 
be. 

And their documents — and this is Exhibit 53 where they 
talk about the next year at NFPA. This was NFPA's previous 
president, was talking about the need to change the business 
model anyway because of technological advances. So asking the 
Court to defend this business model is not an appropriate factor 
to take into consideration when their business model has to 
change anyway, and evolution of business models is natural. 

You know, there had to be an evolution of business models 
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for the Southern Building Codes Conference after Veeck . Lexis 
changed West's business model. Google Scholar is changing 
Lexis's business model. Everybody adapts. PACER has threatened 
the business model of the courthouse filing and retrieval 
systems. 

Business models evolve, and there's no hardship to say, oh, 
well, our business model may have to evolve. The failure of 
plaintiffs to exert a monopoly power over the law is not itself 
a hardship that the Court should take into account. 

I want to go back again to one of the experts for AERA, 

Mr. Geisinger. Complete whiff on the ascription of losses 
because he got the years wrong. He got the years wrong. 

Public.Resource didn't start posting standards till two years 
into the catastrophic decline. When an expert has such a bad 
mistake on the key fact for which he keeps getting cited, it is 
just not evidence. So the hardship is not there. 

Let's talk about the hardship to Public.Resource. 

No, there would be no financial effect on Public.Resource, but 
Public.Resource has a mission, and that mission is to make the 
law accessible to every American: poor Americans as well as rich 
Americans, disabled Americans as well as abled Americans. And 
one of the things it does is make it possible for all sorts of 
Americans to do things with the law that bring power to persons 
to analyze the law, to critique it, to run their data analysis 
tools on it because of the way they are implemented. 
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There is no other way for Public.Resource to make these 
public tools available other than by doing what it's doing. 

So there would be a hardship. Not a financial hardship, but it 
would be a hardship to the very beneficial mission of 
Public.Resource. 

So that takes us to the public interest, and there is a 
very broad public interest here. Now, I think that the 
plaintiffs tend to think of their communities as people engaged 
in building or designing or law enforcement or law making. If 
you look at all the stakeholders who come together, these are 
people who are sort of their community. They're not so focused 
on all the public. 

I mean, certainly they care about public safety; we grant 
that. But they're not sort of — they're not available to 
people to try to sort of stick their nose in and find out, well,, 
what's going on with the law making here? What's going on with 
the regulations that apply to my child's school or to my child's 
safety seat? 

They require — for access, by the way, they require — and 
I went to the NFPA site. I wanted to see — I couldn't do it. 
Because for me to go get that public access, I had to agree to 
consent to jurisdiction of the states where they're located. 

I had to enter into a contract, and I had to acknowledge their 
copyright as a matter of contract in order to have access to 
their public reading rooms. 
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So the fact that I have to enter into a contract, I have to 
submit to jurisdiction of a distant court? That's not real 
public access. That's exactly what they want. It's our 
control, our control over the law itself, and that is a problem. 

We have a problem, Your Honor. I'm not sure I want to say 
it's a problem. It's a controversy right now over the 
privatization of public functions. We've got private operators 
of federal prisons and immigration facilities — 

THE COURT: Keep your argument, though, to the issue 
of remedies, because we are really running out of time. 

MR. BRIDGES: But I think the question is, is a remedy 
at all important? And my point is the public interest would be 
disserved by an injunction that more allocates to the plaintiffs 
a private right over controlling access to the law. They have 
said it's loss of control. 

They have said they have a power to exclude. That's fine 
when it's just an ordinary copyrighted work. It's not fine when 
they are claiming — and the phrase is in their briefs: loss of 
control, power to exclude. When they are claiming a power to 
exclude anybody from the law, for any reason, that is not in the 
public interest. 

The public interest is in having no private gatekeepers to 
the law, because what everyone thinks about emergency managers 
in Michigan or privatization of parking meters in Chicago, 
privatizing the law and giving any private party exclusive 
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control and the power to exclude what anybody chooses to do with 
the law and, oh, maybe it's only $49. That's still saying, your 
right to do what you want to do with the law? Pay us $49, and 
it's all yours. This is unconscionable. Your Honor. 

THE COURT: Or go to the library and make a photocopy. 

MR. BRIDGES: Your honor, I'm not sure that works for 
someone in Helena, Montana, or Anaconda, Montana. His statement 
about accessibility in libraries, it doesn't pan out. There is 
one specific version that is incorporated into law, and that's 
not — his statistics were not right about the specific version. 
And these are not available widely in public libraries. They 
aren't. 

One of the interesting things, a Polish graduate student 
about Polish law asked them and said, I want to quote this 
standard in my thesis. I want to quote this standard in my 
thesis, and my thesis will only go to the three people on my 
thesis committee. And they said, Sorry. You can't. You'll 
just have to cite to it. 

This is control. And when it becomes the law, ordinary 
control of a copyright holder over a copyrighted work, I get 
that, but not when it becomes the law. Your Honor. 

THE COURT: Thank you, Mr. Bridges. 

MR. BRIDGES: Thank you. 

THE COURT: I have to walk out of this courtroom in 
three minutes. All right? 
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MR. HUDIS: Real fast. 

THE COURT; The chief judge is waiting for me, and 
that's somebody I'm not going to keep waiting. 

MR. HUDIS: I want to make sure we get this in the 
record. Your Honor. Dr. Geisinger did not whiff. He got it 
right on our present harm. It's submitted into the record, 
paragraphs 24 through 27 of his declaration. He got it right. 
And Mr. Bridges can pontificate all he wants. We have shown 
harm. We've shown not only past harm but also likelihood of 
irreparable future harm. Thank you. Your Honor. 

THE COURT: All right. Thank you very much. 

Thank you all for your very hard work and your real effort in 
presentation and your arguments, which were very well prepared. 
Thank you. 

(Proceedings adjourned at 12:57 p.m.) 
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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


) 

AMERICAN SOCIETY FOR TESTING ) 

AND MATERIALS, et al., ) 

) 

Plaintiffs, ) 

) Case No. 13-cv-1215 (TSC) 
v. ) 

) 

PUBLIC.RESOURCE.ORG, INC., ) 

) 

Defendant. ) 

) 

AMERICAN EDUCATIONAL ) 

RESEARCH ASSOCIATION, INC. et al, ) 

) 

Plaintiffs, ) 

) 

v. ) Case No. 14-cv-0857 (TSC) 

) 

PUBLIC.RESOURCE.ORG, INC., ) 

) 

Defendant. ) 

_ ) 

MEMORANDUM OPINION 


Before the court are motions and cross-motions for summary judgment in two related 
cases. Because there is significant factual and legal overlap between the two cases, the court 
issues this consolidated opinion to be filed in both cases. 

Plaintiffs American Society for Testing and Materials (“ASTM”), National Fire 
Protection Association, Inc. (“NFPA”), and American Society of Heating, Refrigerating, and 
Air-Conditioning Engineers (“ASHRAE”) (collectively “ASTM Plaintiffs”) brought suit against 
Defendant Public.Resource.org, Inc. (“Public Resource”) under the Copyright Act (17 U.S.C. 

§ 101 etseq .) and the Lanham Act (15 U.S.C. § 1051 et seq.), alleging copyright infringement 
and trademark infringement. Plaintiffs American Educational Research Association, Inc. 
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(“AERA”), American Psychological Association, Inc. (“APA”), and National Council on 
Measurement in Education, Inc. (“NCME”) (collectively “AERA Plaintiffs”) also brought 
copyright infringement claims against Public Resource under the Copyright Act. Plaintiffs 1 in 
both cases seek permanent injunctions barring Defendant from continued display of their works. 

Plaintiffs moved for summary judgment, and Defendant filed cross-motions for summary 
judgment in both cases. The court held a combined oral argument on September 12, 2016 to 
consider the motions. Upon consideration of the parties’ filings, the numerous amicus briefs, 
and the arguments presented at the motions hearing, and for the reasons stated herein, the ASTM 
Plaintiffs’ motion for summary judgment is GRANTED and Defendant’s cross-motion is 
DENIED. The AERA Plaintiffs’ motion for summary judgment is GRANTED IN PART AND 
DENIED IN PART, and Defendant’s cross-motion is DENIED. 

I. FACTUAL BACKGROUND 

A, The Parties 

1. ASTM Plaintiffs 

ASTM Plaintiffs are not-for-profit organizations that develop private sector codes and 
standards in order to advance public safety, ensure compatibility across products and services, 
facilitate training, and spur innovation. (See ASTM Pis. Statement of Material Facts (“PSMF”) 

HI 9, 13, 14, 86, 87, 129, 130 (ASTM ECF No. 118-2)). 2 These standards include technical 
works, product specifications, installation methods, methods for manufacturing or testing 
materials, safety practices, and other best practices or guidelines. (Id. ][ I ). ASTM has 


1 For simplicity, the court’s use of “Plaintiffs” refers collectively to the ASTM Plaintiffs and 
AERA Plaintiffs. 

2 All initial citations to the record in this Opinion will include the docket number as “ASTM 
ECF” or “AERA ECF.” 
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developed over 12,000 standards that are used in a wide range of fields, including consumer 
products, iron and steel products, rubber, paints, plastics, textiles, medical services and devices, 
electronics, construction, energy, water, and petroleum products, and are the combined efforts of 
over 23,000 technical members, representing producers, users, consumers, government, and 
academia. (Id. ]fl| 13, 28, 41). NFPA has developed over 300 standards in the areas of fire, 
electrical, and building safety, with the goal of reducing the risk of death, injury, and property 
and economic loss due to fire, electrical, and related hazards. {Id. 86, 87, 92). NFPA’s most 
well-known standard is the National Electrical Code, first published in 1897 and most recently in 
2014. {Id. ]fl[ 93-94). Finally, ASHRAE has published over 100 standards for a variety of 
construction-related fields, including energy efficiency, indoor air quality, refrigeration, and 
sustainability. {Id. | 130). 

2. AERA Plaintiffs 

AERA Plaintiffs are not-for-profit organizations that collaboratively develop the 
Standards for Educational and Psychological Testing, including the 1999 edition at issue in this 
case (“the 1999 Standards”). (AERA PSMF fflf 1, 5, 13 (AERA ECF No. 60-2)). AERA is a 
national scientific society whose mission is “to advance knowledge about education, to 
encourage scholarly inquiry related to education, and to promote the use of research to improve 
education.” {Id. ][ 2). APA is the world’s largest association of psychologists, and its mission is 
“to advance the creation, communication, and application of psychological knowledge.” {Id. 

]j 3). Finally, NCME is a professional organization “for individuals involved in assessment, 
evaluation, testing, and other aspects of educational measurement.” (Id. ][ 4). 

3. Public Resource 

Defendant Public Resource is a not-for-profit entity devoted to publicly disseminating 
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legal information. (ASTM DSMF 1-2 (ASTM ECF No. 120-3); AERA DSMF ffl[ 1-2 
(AERA ECF No. 68-3)). Its mission is “make the law and other government materials more 
widely available so that people, businesses, and organizations can easily read and discuss [the] 
laws and the operations of government.” (ASTM DSMF 12; AERA DSMF 12). Public 
Resource has posted government-authored materials on its website, including judicial opinions, 
Internal Revenue Service records, patent filings, and safety regulations. (ASTM DSMF ^|f 3—4; 
AERA DSMF 3—4). It does not charge fees to view or download the materials on its website. 
(ASTM DSMF f 5; AERA DSMF f 5). 

B. Incorporation by Reference of Industry Standards 

In the United States, a complex public-private partnership has developed over the last 
century in which private industry groups or associations, rather than government agencies, 
typically develop standards, guidelines, and procedures that set the best practices in a particular 
industry. 3 Applicable standards are used by entities and individuals in order to self-regulate and 
conform to the best practices of that industry. Professor Peter Strauss has noted that 
“manufacturing and markets are greatly aided, and consumers offered protection, by the 
application of uniform industrial standards created independent of law, as means of assuring 
quality, compatibility, and other highly desired market characteristics.” Peter L. Strauss, Private 
Standards Organizations and Public Law, 22 Wm. & Mary Bill Rts. J. 497, 499 (2013). 


3 See U.S. Office of Management and Budget, Revised Circular No. A-l 19, 
https://obamawhitehouse.archives.gov/sites/default/files/omb/inforeg/revised_circular_a- 
119_as_of_l_22.pdf (“OMB Revised Circular”) at 1 (Jan. 27, 2016) (“The vibrancy and 
effectiveness of the U.S. standards system in enabling innovation depends on continued private 
sector leadership and engagement. Our approach—reliance on private sector leadership, 
supplemented by Federal government contributions to discrete standardization processes as 
outlined in OMB Circular A-l 19—remains the primary strategy for government engagement in 
standards development.”). 
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Standards are typically developed by standards developing organizations (“SDOs”), like 
Plaintiffs, who work to develop “voluntary consensus standards,” such as those here. Voluntary 
consensus standards are the ultimate product of many volunteers and association members from 
numerous sectors bringing together technical expertise. They are “developed using procedures 
whose breadth of reach and interactive characteristics resemble governmental rulemaking, with 
adoption requiring an elaborate process of development, reaching a monitored consensus among 
those responsible within the SDO.” Id. at 501. ASTM Plaintiffs develop their standards using 
technical committees with representatives from industry, government, consumers, and technical 
experts. (ASTM PSMF 7, 28, 29, 109, 114, 135). These committees conduct open 
proceedings, consider comments and suggestions, and provide for appeals, and through 
subcommittees, draft new standards, which the full committees vote on. (Id. fflf 31-37, 109, 136, 
139). The AERA Plaintiffs developed the 1999 Standards through a Joint Committee which 
considered input from the public in a notice-and-comment process. (AERA PSMF 13-16). 

Pursuant to 5 U.S.C. § 552, federal agencies may incorporate voluntary consensus 
standards—as well as, for example, state regulations, government-authored documents, and 
product service manuals—into federal regulations by reference. See Emily S. Bremer, 
Incorporation by Reference in an Open-Government Age, 36 Harv. J.L. & Pub. Pol’y 131, MS- 
47 (2013) (providing a general overview of the federal government’s incorporation of materials 
by reference). The federal government’s practice of incorporation by reference of voluntary 
consensus standards is intended to achieve several goals, including eliminating the cost to the 
federal government of developing its own standards, encouraging long-term growth for U.S. 
enterprises, promoting efficiency, competition, and trade, and furthering the reliance upon 
private sector expertise. See OMB Revised Circular, supra, at 14. 
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Section 552(a)(1) provides that “a person may not in any manner be required to resort to, 
or be adversely affected by, a matter required to be published in the Federal Register and not so 
published[, but] . . . matter reasonably available to the class of persons affected thereby is 
deemed published in the Federal Register when incorporated by reference therein with the 
approval of the Director of the Federal Register.” 5 U.S.C. § 552(a)(1) (emphasis added). The 
Office of the Federal Register (“OFR”) adopted regulations pursuant to § 552(a)(1) in 1982 and 
issued revised regulations in 2014. See Approval Procedures for Incorporation by Reference, 47 
Fed. Reg. 34,107 (Aug. 6, 1982) (codified at 1 C.F.R. § 51.1 etseq.); 79 Fed. Reg. 66,267 (Nov. 

7, 2014). These regulations specify that a “publication is eligible for incorporation by reference” 
if it is “published data, criteria, standards, specifications, techniques, illustrations, or similar 
material; and [d]oes not detract from the usefulness of the Federal Register publication system.” 

1 C.F.R. § 51.7(a)(2). To determine whether the material is “reasonably available” as required 
by the statute, OFR will consider “[t]he completeness and ease of handling of the publication” 
and “[w]hether it is bound, numbered, and organized, as applicable.” Id. § 51.7(a)(3). All the 
standards at issue in this case have been incorporated by reference into federal law. (ASTM 
DSMF122; 34 C.F.R. § 668.146 (incorporating AERA Plaintiffs’ 1999 Standards). 

Standards that are incorporated by reference are available in person at the OFR in 
Washington, DC and/or with the incorporating agency. See 1 C.F.R. § 51.3(b)(4). Federal 
regulations that incorporate standards by reference typically direct interested individuals or 
entities to location(s) where they may view the incorporated documents in person. For example, 
the Environmental Protection Agency’s (“EPA”) regulation, 40 C.F.R. § 60.17(a), which 
incorporates numerous standards at issue here, states that: 

Certain material is incorporated by reference into this part with the approval of the 

Director of the Federal Register under 5 U.S.C. § 552(a) and 1 CFR part 51. . . . 
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All approved material is available for inspection at the EPA Docket Center, Public 
Reading Room, EPA WJC West, Room 3334, 1301 Constitution Ave. NW, 
Washington, DC, telephone number 202-566-1744, and is available from the 
sources listed below. It is also available for inspection at the National Archives and 
Records Administration (NARA). For information on the availability of this 
material at NARA, call (202) 741-6030 or go to http://www.archives.gov/ 
federalregister/codeoffederalregulations/ibrlocations.html. 

The EPA regulation further specifies that, for example, the 206 ASTM standards incorporated by 

reference by the EPA (some of which are involved in this suit) are “available for purchase from 

ASTM International, 100 Barr Harbor Drive, P.O. Box CB700, West Conshohocken, 

Pennsylvania 19428-2959, (800) 262-1373, http://www.astm.org.” 40 C.F.R. § 60.17(h). The 

U.S. Department of Education incorporated the AERA Plaintiffs’ 1999 Standards by reference at 

34 C.F.R. § 668.146(b)(6), which states that the standards are: 

on file at the Department of Education, Federal Student Aid, room 113E2, 830 First 
Street, NE, Washington, DC 20002, phone (202) 377-4026, and at the National 
Archives and Records Administration (NARA). For information on the availability 
of this material at NARA, call 1-866-272-6272, or to go: http://www.archives.gov/ 
federal-register/code-of-federal-regulations/ibr-locations.html. The document may 
also be obtained from the American Educational Research Association. 

ASTM Plaintiffs sell PDF and hard copy versions of their standards, including those that 

have been incorporated by reference into law. (ASTM PSMF 57, 99, 157). The prices for the 

standards in this case range from $25 to $200. (Id. 58, 99, 158). The ASTM Plaintiffs also 

maintain “reading rooms” on their websites that allow interested parties to view Plaintiffs’ 

standards that have been incorporated by reference. (Id. 63-64, 100, 161). The standards in 

these reading rooms are “read-only,” meaning they appear as images that may not be printed or 

downloaded. (Id.). AERA Plaintiffs sell hardcopy versions of the 1999 Standards, but do not 

sell digital or PDF versions. (AERA PSMF 30, 33). The prices for the 1999 Standards have 

ranged from $25.95 to $49.95 per copy, and they were sold continuously from 2000 through 

2014, except for a nearly two-year period. (Id. 34-35). 

7 


JA3407 


Case l:14-cv-00857-TSC Document 117 Filed 02/02/17 Page 8 of 55 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 386 of 441 

C. Plaintiffs’ Claims in This Action 

1. ASTM et al. v. Public Resource 

This case involves 257 of ASTM Plaintiffs’ standards that have been incorporated by 
reference into federal law. (See ASTM Compl. Ex. A-C; ASTM DSMF ]f 22). Defendant 
admits that it purchased hard copies of each of the standards at issue, scanned them into PDF 
files, added a cover sheet, and posted them online. (ASTM DSMF fflf 173-74, 177-78; ASTM 
PSMF 182-87). Defendant re-typed some of ASTM Plaintiffs’ standards and posted them 
online, with text in Hypertext Markup Fanguage (HTMF) format and graphics and figures in 
Mathematics Markup Fanguage and Scalable Vector Graphics formats. (ASTM DSMF 83, 

175). The copies posted on Defendant’s website all bore ASTM Plaintiffs’ trademarks. (ASTM 
PSMF 1210). Defendant also uploaded the ASTM Plaintiffs’ standards to the Internet Archive, 
a separate independent website. {Id. 1185). 

The ASTM Plaintiffs allege that their standards are original works protected from 
copyright infringement, and brought claims of copyright infringement, contributory copyright 
infringement, trademark infringement, unfair competition and false designation, and trademark 
infringement under common law. (ASTM Compl. 142-95). Defendant counter-sued, seeking 
a declaratory judgment that its conduct does not violate copyright law or trademark law. (ASTM 
Ans. lfl[ 174—205). Both sides have filed motions for summary judgment. 

2. AERA et al. v. Public Resource 

This case involves the 1999 Standards, which AERA Plaintiffs have sold since 2000. 
(AERA PSMF ]j][ 34—35). In May 2012, Public Resource purchased a paper copy of the 1999 
Standards, disassembled it, scanned the pages, created a PDF file, attached a cover sheet, and, 
without authorization from the AERA Plaintiffs, posted the PDF file to Public Resource’s 
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website and the Internet Archive. (AERA DSMF ][ 28; AERA PSMF ][][ 69-80). Public 
Resource posted a read-only version of the 1999 Standards to its website, unlike many of the 
ASTM Plaintiffs’ standards, which had undergone optical character recognition (“OCR”) 
processing to be text-searchable. {Id. 173). OCR processing uses a machine to recognize letters 
and words in a PDF and translate them into letters or words that can be searched and used by 
text-to-speech software for individuals who are blind or visually impaired. (Id. ]|]| 73-75). 

Plaintiffs allege that the 1999 Standards are protected original works, and they brought 
suit claiming copyright infringement and contributory copyright infringement. (AERA Compl. 

Tff 50-63). Defendant counter-sued seeking a declaratory judgment that its conduct does not 
violate copyright law or trademark law. (AERA Ans. 116-37). Both sides have moved for 
summary judgment. 

II. LEGAL STANDARD 

Summary judgment may be granted if “the movant shows that there is no genuine dispute 
as to any material fact and the movant is entitled to judgment as a matter of law.” Fed. R. Civ. P. 
56(a); see also Anderson v. Liberty Lobby, Inc., All U.S. 242, 247^48 (1986) (“[T]he mere 
existence of some alleged factual dispute between the parties will not defeat an otherwise 
properly supported motion for summary judgment; the requirement is that there be no genuine 
issue of material fact.”) (emphasis in original); Holcomb v. Powell, 433 F.3d 889, 895 (D.C. Cir. 
2006). Summary judgment may be rendered on a “claim or defense ... or [a] part of each claim 
or defense.” Fed. R. Civ. P. 56(a). 

“A party asserting that a fact cannot be or is genuinely disputed must support the 
assertion by ... citing to particular parts of materials in the record.” Fed. R. Civ. P. 56(c)(1)(A). 

“A fact is ‘material’ if a dispute over it might affect the outcome of a suit under governing law; 
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factual disputes that are ‘irrelevant or unnecessary’ do not affect the summary judgment 
determination. An issue is ‘genuine’ if ‘the evidence is such that a reasonable jury could return a 
verdict for the nonmoving party.’” Holcomb, 433 F.3d at 895 (quoting Liberty Lobby, 477 U.S. 
at 248) (citation omitted). The party seeking summary judgment “bears the heavy burden of 
establishing that the merits of his case are so clear that expedited action is justified.” Taxpayers 
Watchdog, Inc. v. Stanley, 819 F.2d 294, 297 (D.C. Cir. 1987). 

In considering a motion for summary judgment, “[t]he evidence of the non-movant is to 
be believed, and all justifiable inferences are to be drawn in his favor.” Liberty Lobby, 477 U.S. 
at 255; see also Mastro v. Potomac Elec. Power Co., 447 F.3d 843, 850 (D.C. Cir. 2006) (“We 
view the evidence in the light most favorable to the nonmoving party and draw all inferences in 
its favor.”). The nonmoving party’s opposition, however, must consist of more than mere 
unsupported allegations or denials, and must be supported by affidavits, declarations, or other 
competent evidence setting forth specific facts showing that there is a genuine issue for trial. See 
Fed. R. Civ. P. 56(c); Celotex Corp. v. Catrett, All U.S. 317, 324 (1986). The non-movant “is 
required to provide evidence that would permit a reasonable jury to find [in his favor].” 

Laningham v. U.S. Navy, 813 F.2d 1236, 1242 (D.C. Cir. 1987). 

III. ANALYSIS 

A. Copyright Infringement 

Under the Copyright Act, copyright in a work vests initially in the author(s) of that work. 

17 U.S.C. § 201(a). Ownership can be transferred in whole or in part, and the exclusive rights of 
copyright ownership may also be transferred. Id. § 201(d). An owner of a valid copyright has 
the “exclusive right” to reproduce, distribute, or display the copyrighted works as well as prepare 
derivative works based upon it. Id. § 106(1)—(3), (5). Anyone who violates the exclusive rights 
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of the copyright owner “is an infringer of the copyright or right of the author, as the case may 
be.” Id. § 501(a). The legal or beneficial owner of that exclusive right may then “institute an 
action for any infringement.” Id. § 501(b). In order to succeed on their copyright infringement 
claims, the Plaintiffs must prove both ‘“(1) ownership of a valid copyright, and (2) copying of 
constituent elements of the work that are original.’” Stenograph, LLC v. Bossard Assoc., Inc., 

144 F.3d 96, 99 (D.C. Cir. 1998) (quoting FeistPubl’ns, Inc. v. Rural Tel. Serv. Co., 499 U.S. 

340, 361 (1991)). 

1. Feist Prong 1: Ownership of a Valid Copyright 

a. Ownership 

The court must first decide the threshold issue of whether Plaintiffs own the copyrights in 
part or outright such that they have standing to challenge Defendant’s alleged infringement. The 
Copyright Act provides that possession of a certificate of registration from the U.S. Copyright 
Office “made before or within five years after first publication of the work shall constitute prima 
facie evidence,” creating a rebuttable presumption of ownership of a valid copyright. 17 U.S.C. 

§ 410(c); see also MOB Music Publ’g. v. Zanzibar on the Waterfront, LLC, 698 F. Supp. 2d 197, 
202 (D.D.C. 2010). If the copyright was registered more than five years after the work was 
published, then the “evidentiary weight to be accorded .. . shall be within the discretion of the 
court.” 17 U.S.C. § 410(c). 

When a party offers as prima facie evidence a registration certificate for a compilation of 
individual works that it authored, rather than the registration for a specific individual work, a 
court may consider this to be similar prima facie evidence of ownership, creating the same 
rebuttable presumption. SeeXoom, Inc. v. Imageline, Inc., 323 F.3d 279, 283-84 (4th Cir. 2003), 
abrogated by Reed Elsevier, Inc. v. Muchnick, 559 U.S. 154 (2010); Morris v. Business 
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Concepts, Inc., 259 F.3d 65, 68 (2d Cir. 2001), abrogated on other grounds by Muchnick, 559 
U.S. 154 (2010). Moreover, the registration certificate is sufficient prima facie evidence for the 
individual works within the compilation if the compilation is deemed to be a “single work.” 

Federal regulations provide that “all copyrightable elements that are otherwise recognizable as 
self-contained works, that are included in a single unit of publication, and in which the copyright 
claimant is the same” constitute a “single work,” such that they are validly registered under a 
single registration certificate 37 C.F.R. § 202.3(b)(4)(A); Kay Berry, Inc. v. Taylor Gifts, Inc., 

4221 F.3d 199, 205-06 (3d Cir. 2005); Yurman Studio, Inc. v. Castaneda, 591 F. Supp. 2d 471, 

483 (S.D.N.Y. 2008). 

Once a copyright holder has proffered this prima facie evidence, the alleged infringer 
“challenging the validity of the copyright has the burden to prove the contrary.” Hamil Am., Inc. 
v. GFI, Inc., 193 F.3d 92, 98 (2d Cir. 1999); United Fabrics Int’l, Inc. v. C&J Wear, Inc., 630 
F.3d 1255, 1257 (9th Cir. 2011) (infringer “has the burden of rebutting the facts set forth in the 
copyright certificate”). The defendant-infringer might argue that the plaintiff-copyright holder 
had some defect in the record-keeping submitted to establish ownership. However, this “skips a 
step,” as the defendant must first “set forth facts that rebut the presumption of validity to which 
[the plaintiffs] copyright is entitled” before attacking the sufficiency of a plaintiffs evidence of 
ownership. United Fabrics, 630 F.3d at 1257. The infringer must use “ other evidence in the 
record [to] cast[] doubt on” the validity of the ownership. Fonar Corp. v. Domenick, 105 F.3d 
99, 104 (2d Cir. 1997) (emphasis in original). The court in Fonar noted that defendant-infringers 
have overcome the presumption of validity with evidence that the work has been copied from the 
public domain and evidence that the work was non-copyrightable. Id. (citing Folio Impressions, 
Inc. v. Byer Cal., 937 F.2d 759, 763-64 (2d Cir. 1991); Carol Barnhart, Inc. v. Economy Cover 
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Corp., 773 F.2d 411, 414 (2d Cir. 1985)). Parties challenging the validity of copyright 
registrations must therefore do more than simply point out potential errors in the certificate. See 
2 Nimmer on Copyright § 7.20(b)(1) (“a misstatement... in the registration application, if 
unaccompanied by fraud, should neither invalidate the copyright nor render the registration 
certificate incapable of supporting an infringement action”). 

The ASTM Plaintiffs produced copyright certificates for each of the nine standards at 
issue, and each of these certificates list the ASTM Plaintiffs as the authors of the works. 4 The 
AERA Plaintiffs also produced the copyright certificates for the 1999 Standards, listing the 
AERA Plaintiffs as authors. 5 Two of ASTM’s standards—D86-07 and D975-07—were 
registered more than five years after they were published. The court accords these the same 
evidentiary weight as if they had been registered within five years. See 17 U.S.C. § 410(c) (court 
has discretion over evidentiary weight). Moreover, the court finds that the registration certificate 
for the 1999 Book of Standards sufficiently establishes prima facie evidence of ASTM’s 
ownership of D396-98 and D1217-93(98). Therefore, the ASTM Plaintiffs and AERA Plaintiffs 
have established their ownership of the works at issue with prima facie evidence. 


4 The nine copyright registrations are provided in the record here: 

■ ASTM : Ex. 1 to O’Brien Decl. (ASTM D86-07) (ASTM ECF No. 118-7, p. 13); Ex. 2 to 
O’Brien Decl. (ASTM D975-07) (ASTM ECF No. 118-7, p. 16); Ex. 4 to O’Brien Decl. 
(1999 Annual Book of ASTM Standards) (ASTM ECF No. 118-7, p. 23); Ex. 3 to O’Brien 
Decl. (listing ASTM D396-98 and ASTM D1217-93(98) as standards included in the 1999 
Annual Book of ASTM Standards) (ASTM ECF No. 118-7, pp. 20-21). 

■ NFPA : Ex. A to Berry Decl. (National Electrical Code, 2011 ed.) (ASTM ECF No. 118-3, 
p. 6); Ex. B to Berry Decl. (2014 ed.) (ASTM ECF No. 118-3, p. 8). 

■ ASHRAE : Ex. 3 to Reiniche Decl. (Standard 90.1, 2004 ed.) (ASTM ECF No. 118-10, 
page 16); Ex. 4 to Reiniche Decl. (2007 ed.) (ASTM ECF No. 118-10, page 19); Ex. 5 to 
Reiniche Decl. (2010 ed.) (ASTM ECF No. 118-10, page 22). 

5 Ex. RRR to Levine Decl. (original copyright registration) (AERA ECF No. 60-83); Ex. SSS to 
Levine Decl. (2014 corrected registration) (AERA ECF No. 60-84). 
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The burden to offer evidence disproving ownership thus shifts in both cases to Defendant. 
See Zanzibar, 698 F. Supp. 2d at 202; Roeslin v. District of Columbia, 921 F. Supp. 793, 797 
(D.D.C. 1995) (finding that because the copyright registration listed plaintiff as the author, the 
“burden is thus on the defendant to establish” that plaintiff was not the author). To rebut the 
presumption of validity, in both cases Defendant pointed to the fact that the certificates state that 
the standards were “works for hire”—i.e., that Plaintiffs acquired authorship and ownership 
rights because their employees or anyone who signed a work-for-hire agreement wrote the 
standards—and the certificates further state that Plaintiffs are the authors of the “entire text[s],” 
when Plaintiffs have said that the standards are drafted by hundreds or thousands of volunteer 
contributors. Defendant contends that the certificates must list all of these hundreds or thousands 
of authors in order to be accurate, and that the failure to do so is a material error which strips 
Plaintiffs of the presumption of ownership. However, Defendant offers scant support for this 
argument. 

Moreover, Defendant failed to meet its initial burden, since it did not adduce any 
additional evidence disproving Plaintiffs’ authorship. Instead, Defendant points to weaknesses 
in the additional evidence that Plaintiffs proffered to establish their ownership, including 
questioning whether every one of the hundreds of Plaintiffs’ members who contributed to the 
standards at issue signed an agreement with appropriate language transferring or assigning 
copyright ownership to Plaintiffs. Because Plaintiffs may have standing to bring this 
infringement suit even as part owners of the copyrights, it is not clear why Defendant asserts that 
Plaintiffs must prove outright ownership of their copyrights. Beyond showing that Plaintiffs’ 
recordkeeping could perhaps be more thorough, Defendant has not identified any evidence that 
either the ASTM Plaintiffs or AERA Plaintiffs do not own the copyrights of the standards, in 
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whole or in part. The court therefore concludes that the ASTM Plaintiffs and AERA Plaintiffs 
are the owners of the copyrights at issue and have standing to bring their claims. 6 
b. Valid Copyrights 

Defendant also argues that Plaintiffs do not own “valid” copyrights under Feist because 
the standards either were never copyrightable or lost their copyright protection upon 
incorporation by reference into federal regulations. Defendant argues that the standards cannot 
be copyrighted because: (1) they are methods or systems, which are not entitled to protection 
under 17 U.S.C. § 102(b); (2) the standards are in the public domain as “the law”; and (3) the 
merger and scenes a faire doctrines preclude a finding of infringement. 

(i). Methods or Systems under Copyright Act § 102(b) 

Section 102(b) of the Copyright Act specifies eight types of works that are not protected 
by copyright: “In no case does copyright protection for an original work of authorship extend to 
any idea, procedure, process, system, method of operation, concept, principle, or discovery, 
regardless of the form in which it is described, explained, illustrated, or embodied in such work.” 

17 U.S.C. § 102(b). Though these eight types of works are not further defined in the statute, the 
legislative history accompanying the Copyright Act of 1976 offers some starting guidance: 

“Section 102(b) in no way enlarges or contracts the scope of copyright protection under the 
present law. Its purpose is to restate, in the context of the new single Federal system of 
copyright, that the basic dichotomy between expression and idea remains unchanged.” H.R. 


6 Defendant did not dispute that “ASTM has copyright registrations that cover each of the 
standards at issue in this litigation” except as to one standard, ASTM D323-58(68). (See Def. 
Statement of Disputed Facts 170 (ASTM ECF No. 121-3)). Therefore, unless Defendant 
presents evidence disproving ownership, the court is likely to conclude, based on these copyright 
registrations, that the ASTM Plaintiffs are the owners of the remaining standards at issue in this 
litigation, with the exception of D323-58(68). As to this standard, ASTM will need to present 
additional evidence establishing ownership. 
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Rep. No. 94-1476, at 57, reprinted in 1976 U.S.C.C.A.N. 5659, 5670 (Sept. 3, 1976); S. Rep. 

No. 94-473 (Nov. 20, 1975); see also 1-2A Nimmer on Copyright § 2A.06(a)(l) (summarizing 
legislative history). The “basic dichotomy” refers to the well-established principle that ideas 
ca nn ot be copyrighted, but expression of those ideas can be. See 1-2A Nimmer on Copyright 
§ 2 A. 06(a)(2)(b) (a work “is to be denied protection only if that protection would be tantamount 
to protecting an excluded category (e.g., idea or method of operation) without regard to the fact 
that the excluded subject matter is expressed or embodied in expression”). 

This section of the Copyright Act codifies the Supreme Court’s 1879 decision in Baker v. 
Selden, 101 U.S. 99 (1897), which denied copyright protection for systems, methods, processes, 
and ideas. Baker evaluated a copyright claim by the author of a manual describing “a peculiar 
system of book-keeping” against a defendant who published a similar guide to book-keeping 
using “a similar plan so far as results are concemed[,] but mak[ing] a different arrangement of 
the columns, and us[ing] different headings.” Id. at 100. The Court defined the question as 
“whether the exclusive property in a system of book-keeping can be claimed, under the law or 
copyright, by means of a book in which that system is explained.” Id. at 101. In answering this 
question, the Court offered as an example that “[t]he copyright of a work on mathematical 
science cannot give to the author an exclusive right to the methods of operation which he 
propounds, or to the diagrams which he employs to explain them, so as to prevent an engineer 
from using them whenever occasion requires.” Id. at 103. This distinction between the actual 
method or system described by a work, which cannot be copyrighted, and the written words 
describing it, which can, is fundamental to understanding the Copyright Act’s modem limitations 
to copyright protection in § 102(b). 

Defendant primarily argues that the Plaintiffs’ standards are completely devoid of 
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creative expression and are merely recitations of processes or procedures that a person or entity 
would follow. Part of this argument appears to rest only on the fact that the names of the ASTM 
Plaintiffs’ standards, and their descriptions or advertisements, include the words “method” and 
“procedure.” See, e.g., ASTM D86-07 Standard Test Method for Distillation of Petroleum 
Products at Atmospheric Pressure, Ex. 6 to Decl. of Thomas O’Brien (“O’Brien Decl.”) (ASTM 
ECF No. 118-7 at 107)); ASTM D1217-93(98) Standard Test Method for Density and Relative 
Density (Specific Gravity) of Liquids by Bingham Pycnometer, Ex. 9 to O’Brien Decl. (ASTM 
ECF No. 118-7 at 136). Additionally, the AERA Plaintiffs’ Rule 30(b)(6) representative noted 
that the 1999 Standards “describe procedures, statistical procedures, research procedures .. . how 
to design a test, how to collect evidence of validity, [and] how to calculate the reliability of 
tests.” (Def. Br. at 32 (citing AERA DSMF ][ 77)). However, simply calling a work a 
“procedure” or a “method” does not revoke its copyright protection under the Copyright Act. 

This argument misunderstands or ignores the expression/idea dichotomy rooted in Baker and 
codified in § 102(b). 

Defendant also emphasizes that because the Plaintiffs’ standards are highly technical, 
complex, and precise, and because testimony shows that the ASTM Plaintiffs attempt to create 
the “best” standards, then the standards are “dictated by utility” or just “discovered facts,” and 
lack any creative expressive content. However, the court rejects the argument that voluntary 
consensus standards, such as those here, are analogous to a list of ingredients or basic 
instructions in a recipe, or a series of yoga poses, as in the cases cited by Defendant. Not only is 
there a vast gulf between the simplicity of an ingredient list and the complexity of the standards, 
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but, more importantly, the standards plainly contain expressive content. 7 As one example, 

ASTM D1217-93 lists under the heading “Significance and Use”: “Although [the standard] is no 
longer employed extensively for the purpose, this test method is useful whenever accurate 
densities of pure hydrocarbons or petroleum fractions with boiling points between 90 and 110°C 
are required.” (ASTMECFNo. 118-7 at 136). 

The standards in these cases contain expression that is certainly technical but that still 
bears markings of creativity. As the Supreme Court instructed in Feist, “the requisite level of 
creativity is extremely low; even a slight amount will suffice. The vast majority of works make 
the grade quite easily, as they possess some creative spark, ‘no matter how crude, humble or 
obvious’ it might be.” 499 U.S. at 345 (quoting 1 M. Nimmer & D. Nimmer, Copyright 
§ 1.08(C)(1) (1990)). Moreover, as Defendant conceded, there are many possible forms of 
expression through which the technical material in the standards could be conveyed, and the 
volunteer and association members who collectively author the standards “debate wording in the 
standards.” (Def. Br. at 32 (ASTM ECF No. 121)). Thus, however “humble” or “obvious” 
Defendant finds the Plaintiffs’ creative choices, the standards still bear at least the “extremely 
low” amount of creativity required by the Supreme Court. Moreover, the undisputed record 
evidence also shows that other parties have written different standards on the same exact subject 
matter as ASTM Plaintiffs’ standards, undermining the argument that the standards are so 
technical and precise there can be only one possible expression. (ASTM PSMF ffl[ 38, 133). 

Importantly, Baker and § 102(b) bar Plaintiffs from attempting to copyright the system or 


7 Defendant does not request that this court scour the over 1,000 pages of the nine of ASTM 
Plaintiffs’ standards provided to the court or the over 200 pages of the 1999 Standards, and the 
court was not provided with copies of the remaining standards. The court declines to engage in 
such an exercise here. 
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method itself, not the written work explaining or describing that method. Here, the copyright 
protections held by the Plaintiffs do not prevent any person or entity from using or applying the 
procedures described in the standards, only from copying their written descriptions of those 
standards. Defendant presented no evidence that the Plaintiffs have sought to block an entity or 
person from using the procedures described in the standards. In fact, use of the procedures 
described is the entire purpose of such voluntary consensus standards. The court therefore 
concludes that § 102(b) of the Copyright Act does not preclude these standards from being 
copyrighted. 

(ii). Loss of Copyright Upon Entering the Public Domain 
A. Federal Law Does Not Bar Copyrightability 
At the heart of Defendant’s defense is the argument that Plaintiffs’ standards lost their 
copyright protections the instant they were incorporated by reference into federal regulations. 

There are weighty policy arguments on both sides of this issue, including the need to preserve a 
vital and complicated public-private partnership between the government and SDOs, and the 
need for an informed citizenry to have a full understanding of how to comply with the nation’s 
legal requirements. However, this suit is not about access to the law in a broad sense, but instead 
about the validity of copyrights for these standards under current federal law. Copyright 
protection is a creature of statute, and as such is the result of careful policy considerations by 
Congress. In the view of this court, Congress has already passed on the question of revoking 
copyright protection for standards that have been incorporated by reference into regulations, and 
any further consideration of the issue must be left to Congress for amendment. 

Section 105 of the Copyright Act states that “[cjopyright protection under this title is not 
available for any work of the United States Government.” 17 U.S.C. § 105. The Act defines a 

19 


JA3419 



Case l:14-cv-00857-TSC Document 117 Filed 02/02/17 Page 20 of 55 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 398 of 441 


“work of the United States Government” as “a work prepared by an officer or employee of the 

United States Government as part of that person’s official duties.” Id. § 101. These are the only 

government-related works that outright lack copyright under the law. For other types of works, 

such as those commissioned by the government or created under government contract by private 

parties, Congress chose to make case-by-case decisions and leave the determination of whether 

private copyright should exist to the federal agency that commissioned or contracted for the 

work. The House Report accompanying the Copyright Act states: 

The bill deliberately avoids making any sort of outright, unqualified prohibition 
against copyright in works prepared under Government contract or grant. There 
may well be cases where it would be in the public interest to deny copyright in the 
writings generated by Government research contracts and the like; it can be 
assumed that, where a Government agency commissions a work for its own use 
merely as an alternative to having one of its own employees prepare the work, the 
right to secure a private copyright would be withheld. However, there are almost 
certainly many other cases where the denial of copyright protection would be 
unfair or would hamper the production and publication of important works. 

Where, under the particular circumstances, Congress or the agency involved finds 
that the need to have a work freely available outweighs the need of the private 
author to secure copyright, the problem can be dealt with by specific legislation, 
agency regulations, or contractual restrictions. 

H.R. Rep. No. 94-1476, at 5672 (1976), reprinted in 1976 U.S.C.C.A.N. 5659, 5672. 

Defendant argues that Sections 102(b) (no protection for systems or methods) and 105 

(no protection for Government-authored works) should be read together to indicate that Congress 

intended that there be no copyright protections for incorporated standards because, like judicial 

opinions—which the Supreme Court nearly two hundred years ago determined could not be 

copyrighted—the standards, once incorporated, are “legal facts” which cannot be copyrighted. 

See Wheaton v. Peters, 33 U.S. 591, 668 (1834) (writing that the Court was “unanimously of the 

opinion that no reporter has or can have any copyright in the written opinions delivered by this 

Court”); Banks v. Manchester, 128 U.S. 244, 253 (1888) (“The whole work done by the judges 
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constitutes the authentic exposition and interpretation of the law, which, binding every citizen, is 
free for publication to all, whether it is a declaration of unwritten law, or an interpretation of a 
constitution or a statute.”). While these cases form the bedrock for the long-standing principle 
that works authored by government officials or employees cannot be copyrighted, the cases 
involved works by actual government officials—i.e., judges—acting in their official capacity, 
unlike here. That was the principle codified in § 105 of the Copyright Act and restated in the 
U.S. Copyright Office’s Compendium of Copyright Office Practices § 313.6(c)(2) (3d ed. 2014), 
which states: “As a matter of longstanding public policy, the U.S. Copyright Office will not 
register a government edict that has been issued by any state, local, or territorial government, 
including legislative enactments, judicial decisions, administrative rulings, public ordinances, or 
similar types of official legal materials.” 

Congress was well aware of the potential copyright issue posed by materials incorporated 
by reference when it crafted Section 105 in 1976. Ten years earlier, Congress had extended to 
federal agencies the authority to incorporate private works by reference into federal regulations. 

See Pub. L. No. 90-23, § 552, 81 Stat. 54 (1967) (codified at 5 U.S.C. § 552) (providing that 
“matter reasonably available to the class of persons affected thereby is deemed published in the 
Federal Register when incorporated by reference therein with the approval of the Director of the 
Federal Register”). However, in the Copyright Act of 1976, Congress made no mention of these 
incorporated works in § 105 (no copyright for “any work of the United States Government”) or 
any other section. As the House Report quoted above indicates, Congress already carefully 
weighed the competing policy goals of making incorporated works publicly available while also 
preserving the incentives and protections granted by copyright, and it weighed in favor of 
preserving the copyright system. See H.R. Rep. No. 94-1476, at 60 (1976) (stating that under 
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§ 105 “use by the Government of a private work would not affect its copyright protection in any 
way”); see also M.B. Schnapper v. Foley, 667 F.2d 102, 109 (D.C. Cir. 1981) (analyzing 
Copyright Act and holding that “we are reluctant to cabin the discretion of government agencies 
to arrange ownership and publication rights with private contractors absent some reasonable 
showing of a congressional desire to do so”). 

However, recognizing the importance of public access to works incorporated by reference 
into federal regulations, Congress still requires that such works be “reasonably available.” 5 
U.S.C. § 552(a)(1). Under current federal regulations issued by the Office of the Federal 
Register in 1982, a privately authored work may be incorporated by reference into an agency’s 
regulation if it is “reasonably available,” including availability in hard copy at the OFR and/or 
the incorporating agency. 1 C.F.R. § 51.7(a)(3). Thirteen years later, Congress passed the 
National Technology Transfer and Advancement Act of 1995 (“NTTAA”) which directed all 
federal agencies to use privately developed technical voluntary consensus standards. See Pub. L. 
No. 104-113, 110 Stat. 775 (1996). Thus, Congress initially authorized agencies to incorporate 
works by reference, then excluded these incorporated works from § 105 of the Copyright Act, 
and, nearly twenty years later, specifically directed agencies to incorporate private works by 
reference. From 1966 through the present, Congress has remained silent on the question of 
whether privately authored standards and other works would lose copyright protection upon 
incorporation by reference. If Congress intended to revoke the copyrights of such standards 
when it passed the NTTAA, or any time before or since, it surely would have done so expressly. 

See Whitman v. Am. Trucking Ass’ns, Inc., 531 U.S. 457, 468 (2001) (“Congress . . . does not 
alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions—it 
does not. . . hide elephants in mouseholes.”); United States v. Fausto, 484 U.S. 439, 453 (1988) 
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(“[It] can be strongly presumed that Congress will specifically address language on the statute 
books that it wishes to change.”). Instead, Congress has chosen to maintain the scheme it created 
in 1966: that such standards must simply be made reasonably available. See 5 U.S.C. 

§ 552(a)(1). 

Moreover, Congress has similarly determined that online access to the nation’s laws and 
regulations need not be provided for no cost. In establishing “a system of online access to the 
Congressional Record [and] the Federal Register,” Congress authorized the Superintendent of 
Documents, under the direction of the Director of the Government Publishing Office, to “charge 
reasonable fees for use of the directory and the system of access.” 44 U.S.C. §§ 4101-02. While 
citing this statute and noting that the Superintendent has chosen not to charge fees for online 
access, OFR in its 2013 proposed rulemaking stated that Congress had not made a policy 
determination that online access to the law must be provided free of charge. See Incorporation 
by Reference, 78 Fed. Reg. 60,784, 60,785 (Oct. 2, 2013). Similarly, OFR recently determined 
that “reasonably available” under § 552(a)(1) did not mean availability for no cost on the 
Internet. See id. (considering proposed amendments to OFR’s regulations on incorporation by 
reference and specifically addressing and rejecting the argument that standards incorporated by 
reference should be posted online for free in order to be reasonably available). 

Importantly, there is no evidence that the ASTM Plaintiffs’ standards or the AERA 
Plaintiffs’ standards are unavailable to the public. In fact, the undisputed record evidence shows 
that the standards are required to be available in physical form from OFR (see 1 C.F.R. 

§ 51.3(b)(4)); are available for purchase from the AERA Plaintiffs in hard copy (AERA PSMF 
134) and from the ASTM Plaintiffs in hard copy and PDFs (see ASTM PSMF 157, 99, 157); 
and are accessible in read-only format for free in ASTM Plaintiffs’ online reading rooms (see 
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ASTM PSMF ]| 64, 100, 161). While Defendant argues that the public requires greater access to 
the standards—in particular, free online access in formats other than read-only—that is a policy 
judgment best left to Congress. The arguments raised by the parties and by amici highlight 
important considerations regarding unrestricted access to the texts of laws, regulations, and 
incorporated materials, as well as the strong need to protect the economic incentives for the 
further creation of new standards through revenues from the sale of existing standards. This is 
the policy balancing that Congress is presumed to have already engaged in, and any further 
changes to the law in light of new technological developments and resulting changes in public 
expectations of access to information are best addressed by Congress, rather than this court. 

B. Due Process Concerns Do Not Bar Copyrightability 
Defendant further argues that even if the Copyright Act does not bar copyright protection 
for incorporated standards, individuals have a due process right to access the text of “the law,” 
including the standards at issue here. Four Circuit Courts have considered similar arguments 
regarding copyrighted works incorporated by reference into state and federal regulations. See 
Bldg. Officials & Code Admins, v. Code Tech., Inc., 628 F.2d 730 (1st Cir. 1980) (“BOCA”) 
(declining to rule on the question); CCCInfo. Servs., Inc. v. McLean Hunter Mkt. Reports, Inc., 

44 F.3d 61, 74 (2d Cir. 1994) (upholding copyright in work incorporated by reference); Cnty. of 
Suffolk, N.Y. v. First Am. Real Estate Solutions, 261 F.3d 179 (2d Cir. 2001) (same); Practice 
Mgmt. Info. Corp. v. Reports, Inc., 121 F.3d 516, 518 (9th Cir. 1997) (same); Veeckv. S. Bldg. 

Code Cong. Int’l, Inc., 293 F.3d 791, 796 (5th Cir. 2002) (en banc) (holding that incorporation 
by reference revoked the copyright owner’s copyright protection). The court will briefly 
describe each of these Circuit decisions. 

The question of whether a privately-authored, copyrighted work might lose its copyright 
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protection after being referenced in a law was first discussed by the First Circuit in BOCA. That 
case involved a nonprofit, BOCA, which authored and copyrighted a model code called the 
“Basic Building Code.” See 628 F.3d at 731-32. Massachusetts adopted a building code based 
in substantial part on the BOCA Basic Building Code, called the Co mm onwealth of 
Massachusetts State Building Code. Id. at 732. BOCA sold a printed version of the 
Massachusetts State Building Code for $22 a copy, and the state referred any persons interested 
in obtaining a copy of the code for their own use to BOCA. Id. The defendant, Code Tech., Inc., 
published its own copy of the Massachusetts State Building Code and sold it for $35 per volume. 

Id. In the subsequent copyright infringement suit, the district court granted BOCA’s request for 
a preliminary injunction, and the First Circuit reversed, though it reserved judgment on the 
merits of whether the building code was validly copyrighted. Instead, it noted that “[t]he citizens 
are the authors of the law, and therefore its owners, regardless of who actually drafts the 
provisions, because the law derives its authority from the consent of the public, expressed 
through the democratic process.” Id. at 734. 

The Second Circuit considered similar issues in two cases. First, in CCC, the court 
considered whether copyright protection for a compilation called the Red Book, which listed 
used car valuations, was revoked after it was referenced by states as one of several references for 
car valuation. See 44 F.3d at 74. The court rejected the argument that referenced works enter 
the public domain, stating: “We are not prepared to hold that a state’s reference to a copyrighted 
work as a legal standard for valuation results in loss of the copyright. While there are indeed 
policy considerations that support [defendant’s public domain] argument, they are opposed by 
countervailing considerations.” Id. The court then analogized to a state education system 
assigning copyrighted books as a mandatory part of a school curriculum and noted that under the 
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public domain logic, these books might lose copyright protection. Id. 

Second, in County of Suffolk, the Second Circuit considered the copyrightability of a 
county’s tax maps. The court looked to Banks, in which the Supreme Court held that judicial 
opinions were not copyrightable, and determined that Banks established two premises: (1) that 
judges’ opinions cannot be copyrighted because judges receive their salaries from the public 
treasury and do not have the economic incentives that copyrights are designed to protect; and 
(2) there are due process considerations because the “whole work done by the judges constitutes 
the authentic exposition and interpretation of the law, which, binding every citizen, is free for 
publication to all.” 261 F.3d at 193-94 (citing Banks v. Manchester, 128 U.S. 244, 253 (1888)). 
Building on these premises, the Second Circuit articulated two factors that should guide courts’ 
analysis in these situations: first, “whether the entity or individual who created the work needs 
an economic incentive to create or has a proprietary interest in creating the work”; and second, 
“whether the public needs notice of this particular work to have notice of the law.” Id. at 194 
(citing Practice Management, 121 F.3d at 518-19; BOCA, 628 F.2d at 734-35). With regard to 
this second factor, the court primarily considered the severity of criminal or civil sanctions 
associated with failure to adhere to the maps at issue. Finding no serious penalties, it focused on 
the fact that citizens had “fair warning” of the tax maps from their reference in the tax statute, 
and there was “no allegation that any individual required to pay the applicable property tax ha[d] 
any difficulty in obtaining access to either the law or the relevant tax map.” Id. at 195. 

Therefore, the maps were entitled to copyright protection. 

Like the Second Circuit, the Ninth Circuit in Practice Management also decided to 
preserve the copyright protections in the American Medical Association’s (“AMA”) publication 
of medical codes and descriptions which had been incorporated by reference by the U.S. Health 
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Care Financing Administration (“HCFA”). Under the HCFA’s regulation, parties seeking health 

insurance reimbursement for Medicare were required to use the codes created and copyrighted 

by the AMA. See 121 F.3d at 518. The Ninth Circuit similarly looked to Banks and focused on 

its premise that there is a due process interest in free access to the law. Like the Second Circuit, 

the court considered this due process interest and ultimately rejected revoking the AMA’s 

copyright because “[tjhere [was] no evidence that anyone wishing to use the [copyrighted codes] 

ha[d] any difficulty obtaining access to it.” Id. at 519. 

Finally, counter to the opinions of other circuits, the Fifth Circuit sitting en banc in Veeck 

focused more heavily on the first Banks premise regarding economic incentives and held that 

copyright protection is revoked when a model code is adopted as law by a municipality, stating 

that “as law, the model codes enter the public domain and are not subject to the copyright 

holder’s exclusive prerogatives.” 293 F.3d at 793. However, the court carefully distinguished its 

decision from the facts in the aforementioned cases. It wrote: 

[T]he limits of this holding must be explained. Several national standards-writing 
organizations joined [defendant] as amici out of fear that their copyrights may be 
vitiated simply by the common practice of governmental entities’ incorporating 
their standards in laws and regulations. This case does not involve references to 
extrinsic standards. Instead, it concerns the wholesale adoption of a model code 
promoted by its author, [defendant], precisely for use as legislation. Caselaw that 
derives from official incorporation of extrinsic standards is distinguishable in 
reasoning and result.... If a statute refers to the Red Book or to specific school 
books, the law requires citizens to consult or use a copyrighted work in the 
process of fulfilling their obligations. The copyrighted works do not ‘become 
law’ merely because a statute refers to them. . . . Equally important, the 
referenced works or standards in CCC and Practice Management were created by 
private groups for reasons other than incorporation into law. To the extent 
incentives are relevant to the existence of copyright protection, the authors in 
these cases deserve incentives.... In the case of a model code, on the other hand, 
the text of the model serves no other purpose than to become law. 

Id. at 803-05. The cases before the court, involving some of the same amici referenced in Veeck, 

do not involve model codes adopted verbatim in their entirety into legislation. Instead, the 
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standards incorporated by reference provide guidelines and procedures that individuals or entities 
must use or reference in the fulfillment of their legal obligations under federal regulations. 

Applying the first premise of Banks to the facts here, Defendant argues that Plaintiffs do 
not require economic incentives to create their standards because they actively lobby and 
advocate for their standards to be incorporated by reference into regulations, including investing 
funds on lobbying to that effect. Therefore, Defendant argues, the court should find that 
Plaintiffs create standards for no purpose other than adoption into law, as the Veeck court 
determined regarding the model code in that case. Here however, the facts indicate that 
Plaintiffs create standards for a wide range of industries, that the majority of their standards are 
not incorporated into regulations, and that even those that have been incorporated by reference 
have undergone updates and revisions to reflect modern use, despite the regulations 
incorporating past versions. Plaintiffs and supporting amici highlight that without copyright 
protection for all of their standards, they will face significant difficulty raising the necessary 
revenue to continue producing high-quality voluntary consensus standards. In its Notice of 
Proposed Rulemaking, OFR relied on this same argument to ultimately reject a proposal to 
require free online access to standards in its “reasonably available” determination. 78 Fed. Reg. 
at 60,785 (“If we required that all materials IBR’d into the CFR be available for free, that 
requirement would compromise the ability of regulators to rely on voluntary consensus 
standards, possibly requiring them to create their own standards, which is contrary to the 
NTTAA and the OMB Circular A-l 19.”). 

As for the second premise of Banks, this court finds that, as in the cases before the 
Second and Ninth Circuits, there is no evidence here that anyone has been denied access to the 
standards by the ASTM Plaintiffs or AERA Plaintiffs. Instead, Defendant simply argues that the 
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public should be granted more expansive access. 

Therefore, considering the Banks holdings and given the existing statutory, regulatory, 
and judicial framework, this court finds that Plaintiffs’ standards have not entered the public 
domain upon their incorporation by reference into federal regulations and do not lose their 
copyright protection. This conclusion does not dismiss or diminish the valid public policy 
concern that citizens benefit from greater access to statutes, regulations, and all materials they 
must reference in fulfilling their legal obligations. The ability to know, understand, and 
communicate the law as a broad concept is of paramount importance to the continued success of 
our democracy. However, changes to the statutory or regulatory framework that reconsider the 
balancing of interests underlying modem copyright law and incorporation by reference must be 
made by Congress, not this court. 

(Hi). Merger Doctrine 

Defendant asks the court to apply the “merger doctrine” to find that the standards cannot 
be copyrighted because the expressions in the standards have merged with the law to become 
facts. Under modem copyright law, there is a well-known dichotomy between “expression,” 
which can generally be copyrighted, and “ideas,” which cannot. 4-13 Nimmer on Copyright 
§ 13.03. The merger doctrine has developed to consider those specific situations in which “the 
idea ‘merges’ with the expression, such that a given idea is inseparably tied to a particular 
expression.” Id. at § 13.03(3). This can occur when there “are so few ways of expressing an 
idea [that] not even the expression is protected by copyright.” Id. (quoting BUC Int’l Corp. v. 

Int’l Yacht Council Ltd., 489 F.3d 1129, 1143 (11th Cir. 2007)). 

The parties disagree as to the proper merger doctrine analysis. Defendant argues that 
upon their incorporation by reference, the standards become “merged” with the “fact” that is the 
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law. Plaintiffs argue that to determine if an idea and expression have merged, the court should 
focus on whether there were any other ways of articulating a particular idea when the work was 
first published, not when it was later incorporated by reference. In essence, the parties disagree 
as to whether the merger doctrine is a question of copyrightability—meaning the Plaintiffs’ 
standards might lose copyright protection upon incorporation by reference—or an affirmative 
defense to copyright infringement—i.e., the allegedly infringing work did not violate copyright 
because there was no other way to express the content of the work. Plaintiffs argue that the 
merger doctrine addresses only the question of copyrightability, and so the court’s analysis 
should focus on whether, at the time the standards were authored, there were no other ways to 
articulate and arrange such standards. Defendant contends that the standards could not be 
expressed any other way after incorporation into regulations, and thus its display of the standards 
was not infringement. 

The court declines to resolve this merger doctrine issue, since under either approach, the 
standards maintain copyright protection. At the time they were authored, there were certainly 
myriad ways to write and organize the text of the standards, and, for the reasons discussed above, 
the standards did not lose their copyright protections upon incorporation by reference into federal 
regulations. Therefore, the merger doctrine neither precludes a finding of copyrightability nor 
serves as a defense for Defendant. 

(iv). Scenes a Faire Doctrine 

Finally, Defendant points to the scenes a faire doctrine, which similarly may be 
approached as a question of copyrightability or an affirmative defense. The doctrine typically 
applies to “incidents, characters, or settings which are as a practical matter indispensable, or at 
least standard, in the treatment of a given topic.” Nimmer § 13.03(4) (quoting Atari, Inc. v. 
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North Am. Philips Consumer Elecs. Corp., 672 F.2d 607, 616 (7th Cir. 1982), cert, denied, 459 
U.S. 880 (1982)). Ni mm er offers examples such as the use of a bar room scene in a film about a 
broken-hearted lover because, as the name of the doctrine suggests, these are “scenes which must 
be done.” Id. Defendant argues here that Plaintiffs’ standards are entirely “uncopyrightable” 
because they are “shaped by external factors,” such as the desire to satisfy regulations and laws 
and to write what Plaintiffs believe to be the most accurate and clear standards. (Tr. of Motions 
Hearing at 62:15-19 (ASTM ECF No. 173); Def. Br. at 34). However, this doctrine is a poor fit 
for Defendant’s arguments. In the court’s view, there is a great deal of difference between every 
detail of the phrasing, explanation, and organization across thousands of pages of standards, 
which Defendant argues is entirely dictated by Plaintiffs’ broad desires for accuracy and clarity, 
and the inclusion of a generic bar room scene in a romantic drama where the audience expects it. 
Defendant offers no cases to support its argument that this doctrine bars copyrightability of the 
standards at issue here, and this court knows of none. The court concludes that the scenes a faire 
doctrine does not act as a bar to the copyrightability of Plaintiffs’ standards and does not serve as 
a defense for Defendant’s display of the standards 

In sum, the court concludes that Plaintiffs own valid copyrights over the standards at 
issue, and that the copyrights were not stripped upon the incorporation by reference into federal 
regulations. 

2. Feist Prong 2: Copying an Original Work 

a. Overview 

Having established that both the ASTM Plaintiffs and AERA Plaintiffs own valid 
copyrights in the standards at issue, the second question for the court under Feist is whether 
Public Resource, by scanning and posting online the standards at issue “cop[ied] anything that 

31 


JA3431 



Case l:14-cv-00857-TSC Document 117 Filed 02/02/17 Page 32 of 55 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 410 of 441 


was ‘original’ to” the Plaintiffs. Feist, 499 U.S. at 361. Copying means exercising any of the 
exclusive rights that 17 U.S.C. § 106 vests in the owners of a copyright. See Call of the Wild 
Movie, LLC v. Does, 770 F. Supp. 2d 332, 351 (D.D.C. 2011). These rights include the rights of 
reproduction, distribution, display, and creation of derivative works. See 17 U.S.C. § 106(1)—(3), 
(5). There is no factual dispute that Public Resource reproduced and posted online for display or 
distribution the standards at issue in this case. Having rejected the application of the merger 
doctrine or scenes a faire doctrine as affirmative defenses, Defendant’s only argument on this 
second prong is therefore that its copying and posting of the standards was “fair use.” 
b. Affirmative Defense of Fair Use 

Under the Copyright Act, fair use of a copyrighted work “is not an infringement of 
copyright.” 17 U.S.C. § 107. Fair use is a defense to a claim of copyright infringement in order 
to “fulfill copyright’s very purpose, ‘to promote the Progress of Science and useful Arts.’” 
Campbell v. Acuff-Rose Music, Inc., 510 U.S. 569, 575 (1994) (quoting U.S. Const, art. I, § 8, cl. 
8). The Copyright Act provides that: 

In determining whether the use made of a work in any particular case is a fair use, 
the factors to be considered shall include— 

(1) the purpose and character of the use, including whether such use is of a 
commercial nature or is for nonprofit educational purposes; 

(2) the nature of the copyrighted work; 

(3) the amount and substantiality of the portion used in relation to the 
copyrighted work as a whole; and 

(4) the effect of the use upon the potential market for or value of the 
copyrighted work. 

17 U.S.C. § 107. The statute further lists examples of uses that are “fair use,” including 
“criticism, comment, news reporting, teaching (including multiple copies for classroom use), 
scholarship, or research.” Id. The fair use doctrine calls for a “case-by-case analysis,” and the 
four statutory factors are meant to provide “general guidance,” weighed together “in light of the 
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purposes of copyright.” Campbell, 510 U.S. at 578-79. 

(i). Purpose and Character of Defendant’s Use of the Standards 

With regard to the first factor, the statute itself offers guidance on the types of purposes 
that might be considered fair use: criticism, commentary, news reporting, teaching, or research. 

Id. § 107. Moreover, the Supreme Court has held that courts should focus on whether the new 
work “supersede^] the objects of the creation ... or instead adds something new, with a further 
purpose or different character, altering the first with new expression, meaning, or message; [the 
question], in other words, [is] whether and to what extent the new work is transformative.” 
Campbell, 510 U.S. at 578-79 (internal quotations omitted). Given the constitutional goal of 
copyright—to promote the development of science and the arts—“the more transformative the 
new work, the less will be the significance of other factors, like commercialism, that may weigh 
against a finding of fair use.” Id. at 579. 

It is undisputed that Public Resource scanned the ASTM Plaintiffs’ standards at issue 
from their physical hardcopies and converted them to searchable PDFs using OCR processing 
(ASTM Pis. SUMF ^ 182) and reproduced some of the standards by re-typing them into HTML 
format. (ASTM PSMF1182; ASTM DSMF | 83). Public Resource scanned the AERA 
Plaintiffs’ 1999 Standards from the physical hard copy and converted them to a PDF file, which 
it then uploaded to its website for display and distribution. (AERA PSMF 69, 71-73; AERA 
DSMF ![ 28). Defendant argues this is transformative in three ways: by providing free access to 
“the law”; by enabling others to use software to analyze the standards; and by enabling those 
with visual impairments to use text-to-speech software. The evidence does not support any of 
these arguments. 

Defendant first argues that it has transformed Plaintiffs’ standards by making identical 
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copies of them and distributing them online for no cost. In Defendant’s view, this is 
transformative because it provides individuals with greater access to “the law.” While Defendant 
argues that its conduct is analogous to those who make copies of copyrighted works in order to 
comply with legal requirements, Defendant was not actually acting to comply with a particular 
law—unlike, for example, an individual who makes a photocopy of the standards located at OFR 
for use on her building project. Instead, Defendant has placed identical copies of Plaintiffs’ 
standards into the online marketplace with no intention to use them itself, but instead to simply 
offer them for free in competition with Plaintiffs’ standards. While Defendant did not earn 
revenue directly from the display of the standards, its activity still bears “commercial” elements 
given that it actively engaged in distributing identical standards online in the same consumer 
market. While this commerciality is not by itself dispositive, it does weigh firmly against fair 
use. See Campbell, 510 U.S. at 594. 

Defendant points to Swatch Group Management Services Ltd. v. Bloomberg L.P., 756 
F.3d 73, 81 (2d Cir. 2014) in support of its proposition that when a copyrighted document is of 
great public importance then posting it online may be transformative. However, Swatch Group 
involved the recording of a private conference call about the company’s earnings report 
involving executives and 132 analysts that Bloomberg then distributed to subscribers of its 
Bloomberg Professional service. Id. at 78-79. Given that Swatch Group instructed call 
participants not to record or broadcast the call, any direct knowledge of what the executives said 
would be limited to those analysts who participated. Id. The facts of Swatch Group do not align 
with those here, where the evidence demonstrates that Plaintiffs’ standards are available to 
anyone for viewing online in ASTM Plaintiffs’ reading rooms, at a public library, at the OFR or 
incorporating agency, or for purchase on Plaintiffs’ websites. This court is unwilling to apply 
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any principles from Swatch Group or similar cases to this case, in which the standards are widely 
available. 

Next, Public Resource argues that distributing the duplicate copies online is 
transformative because, with regard to the ASTM Plaintiffs’ standards, Public Resource first 
altered their formatting through application of OCR or conversion to HTML, which enables 
software analysis or the use of text-to-speech software, and for AERA Plaintiffs’ standards, it 
scanned the hard copy and distributed a PDF version. The court has little difficulty concluding 
that these actions are not transformative. See 4-13 Nimmer on Copyright § 13.05(l)(b); Nihon 
Keizai Shimbun, Inc. v. Comline Bus. Data, Inc., 166 F.3d 65, 72 (2d Cir. 1999) (holding that a 
translation is not a transformative, expressive work); Soc ’y of the Transfiguration Monastery, 

Inc. v. Gregory, 685 F. Supp. 2d 217, 227 (D. Mass. 2010), affirmed, 689 F.3d 29, 59-65 (1st 
Cir. 2012) (“A simple repackaging of a work in a new format, whether on the Internet or on a 
CD-ROM or on a flash drive, is not transformative when the result is simply a mirror image 
reflected on a new mirror.”); see also Authors Guild v. Google, Inc., 804 F.3d 202, 207, 217 (2d 
Cir. 2015) (reasoning Google’s scanning and posting of snippets of copyrighted books online 
was fair use because it made “available information about Plaintiffs’ books without providing 
the public with a substantial substitute for matter protected by the Plaintiffs’ copyright interests 
in the original works or derivatives of them” and added “important value to the basic 
transformative search function, which tells only whether and how often the searched term 
appears in the book”) (emphasis added); Authors Guild, Inc. v. HathiTrust, 755 F.3d 87, 90 (2d 
Cir. 2014) (text searching modification was transformative but where full work was not 
displayed). 

Here, Defendant does not actually perform any analysis on the standards, nor does it offer 
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the service of providing them in an accessible way to those visual impairments. Instead, 

Defendant has identified a series of events that must occur, involving intervening third parties 
and the use of one or more additional software programs, in order for there to be a potentially 
“transformative” use for individuals who are blind or have visual impairments. Defendant in 
both cases proffered the expert report of James Fruchterman, who opined on accessibility of 
written materials for those who are blind. In Fruchterman’s AERA report, he wrote that to make 
a hard copy accessible for those with visual impairments, he would scan the pages, process them 
with OCR to convert the read-only images to searchable text, create a Microsoft Word file, and 
then have it proofread because OCR can create numerous errors. (Expert Rep. of James R. 
Fruchterman at 8 (AERA ECF No. 70-50)). Once such a version is then uploaded online, an 
individual who is blind or visually impaired would then need to use additional screen reader 
software, which “is a program that runs on a personal computer or a smartphone that reads the 
information on the screen aloud (using a computer-synthesized voice) to a blind person.” (Id. at 
3—4). While “most blind people themselves do not have the ability to convert books[,] [s]ome 
blind people have their own home scanners, and if they purchased a used copy online, would be 
able to scan the 1999 Standards page by page on a home scanner, which would take at least two 
hours of labor, and then perform optical character recognition on the title.” (Id. at 8). In his 
ASTM report, Fruchterman wrote that he was able to use a screen reader program to read the text 
of the ASTM Plaintiffs’ standards aloud on Defendant’s website, but not in ASTM Plaintiffs’ 
reading rooms. (Ex. 96 to Becker Deck, Expert Rep. of James R. Fruchterman at 5-7 (ECF No. 
122-6)). Fruchterman noted that some of the PDFs on Defendant’s website were read-only 
images, such as those on ASTM Plaintiffs’ reading rooms, which had to be copied and pasted 
into a Microsoft Word document in order for a screen reader program to operate. (Id. at 16-17). 
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He also noted that individuals who are blind may “independently perform optical character 
recognition on image-based PDFs themselves and access the text that way, and many advanced 
computer users that are blind would be aware that this is possible.” (Id. at 17). He did not opine 
on whether OCR could be performed on the PDFs of standards that ASTM Plaintiffs sell or 
whether he attempted to investigate that as part of his research. 

While it appears Defendant may enable blind individuals, like all other individuals, to 
access the standards at no cost, they still may have to take additional steps like OCR processing 
or converting to a different file type, as well as using additional screen reader programs in order 
to access the standards. There is no evidence that this would not be possible with Plaintiffs’ 

PDFs or by scanning Plaintiffs’ hard copy standards. In Defendant’s view, taking the first step 
or two towards making the standards entirely accessible to those with visual impairments is 
enough to have transformed the standards. This attempts to stretch logic, and certainly the 
doctrine of fair use, too far. Defendant has not offered a sufficiently new purpose to render the 
use transformative, and this weighs against a finding of fair use. 

(ii). Nature of the Copyrighted Standards 

The Supreme Court in Campbell in s tructs that courts should analyze the nature of the 
copyrighted work with “recognition that some works are closer to the core of intended copyright 
protection than others, with the consequence that fair use is more difficult to establish when the 
former works are copied.” 510 U.S. at 586. Many cases create a spectrum between creative, 
fictional expression and factual expression, with the former being “more” protected. See 4-13 
Nimmer § 13.05(A)(2). Defendant argues that Plaintiffs’ standards are “factual,” both because 
they are highly technical and because they are “the law.” However, the Constitution explicitly 
states that copyright exists to “advance the progress of science and the useful arts.” U.S. Const. 
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art. I, § 8, cl. 8. That Plaintiffs’ works involve technical scientific concepts and guidelines does 
not push it away from the core of intended copyright protection, but actually brings it closer. 
Plaintiffs’ standards are vital to the advancement of scientific progress in the U.S. and exactly 
the type of expressive work that warrants full protection under the Constitution and the 
Copyright Act. 

(Hi). Amount and Substantiality of the Portions Defendant Used 
The third factor, “the amount and substantiality of the portion used in relation to the 
copyrighted work as a whole,” 17 U.S.C. § 107(3), weighs overwhelmingly in Plaintiffs’ favor 
and against a finding of fair use. It is undisputed that Defendant copied and distributed identical 
versions of the Plaintiffs’ standards in their entirety. To support its actions as fair use under this 
third factor, Public Resource argues that it was necessary to do so because the full text of the 
standards were incorporated into “the law.” However true it may be that individuals wishing to 
read the text of standards incorporated by reference would want to read them in their entirety, 
this argument is unpersuasive in the fair use analysis. Any market competitor wishing to copy a 
rival’s work and distribute it itself could argue that it “needs” to copy the entire work, otherwise 
its distribution would be less successful. Unsurprisingly, Defendant cannot point to a single case 
that supports its view, and the court finds that this factor also weighs strongly against a finding of 
fair use. 

(iv). Effect of Defendant’s Use Upon Potential Market or Value 
The fourth factor, “the effect of the use upon the potential market for or value of the 
copyrighted work,” 17 U.S.C. § 107(4), “poses the issue of whether unrestricted and widespread 
conduct of the sort engaged in by the defendant would .. . result in a substantially adverse 
impact on the potential market for, or value of, the plaintiffs present work,” 4-13 Nimmer on 

38 


JA3438 


Case l:14-cv-00857-TSC Document 117 Filed 02/02/17 Page 39 of 55 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 417 of 441 

Copyright § 13.05(A)(4); Campbell, 510 U.S. at 589 (quoting Nimmer). Moreover, the analysis 
“must take into account not only of harm to the original but also of harm to the market for 
derivative works.” Campbell, 510 U.S. at 589 (quoting Harper & Row Publishers, Inc. v. Nation 
Enters., 471 U.S. 539, 568 (1985)). When Defendant engages in “mere duplication for 
commercial purposes,” as here, a harm to the potential market for the copyrighted works may be 
inferred. See id. at 590-91. Such an inference is intuitive based on the facts here where 
consumers in the online marketplace are currently presented with the option to purchase a PDF 
or hard copy version of Plaintiffs’ standards directly from them, or may download a PDF of an 
identical standard for no cost. The only logical conclusion is that this choice negatively impacts 
the potential market for Plaintiffs’ standards. 

In Campbell, the Supreme Court noted that “[s]ince fair use is an affirmative defense, its 
proponent would have difficulty carrying the burden of demonstrating fair use without favorable 
evidence about relevant markets.” 510 U.S. at 590. Here, Defendant did not offer expert 
evidence on the economic impact on the markets, instead pointing to testimony by Plaintiffs’ 
executives that they did not track or know of negative impacts thus far on their revenue from 
Defendant’s conduct. This is not enough to overcome the logical presumption that such activity, 
particularly if it became more widespread by others in the marketplace, would impact Plaintiffs’ 
revenues. It is not Plaintiffs’ burden to establish that they have been harmed in the market, but 
Defendant’s burden to affirmatively establish that such conduct could not even “potentially” 
harm the Plaintiffs’ market. Defendant has not done so. 

(v). Overall Assessment 

Whatever merit there may be in Defendant’s goal of furthering access to documents 
incorporated into regulations, there is nothing in the Copyright Act or in court precedent to 
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suggest that distribution of identical copies of copyrighted works for the direct purpose of 
undermining Plaintiffs’ ability to raise revenue can ever be a fair use. The court thus concludes 
that the fair use doctrine does not serve as a valid defense for Defendant’s conduct. 

Therefore, the court finds that the ASTM Plaintiffs’ motion for summary judgment as to 
their copyright infringement claim is GRANTED, and the AERA Plaintiffs’ motion for summary 
judgment as to their copyright infringement claim is also GRANTED. Defendant’s cross¬ 
motions on copyright infringement are both DENIED. 

B. Contributory Copyright Infringement 

AERA Plaintiffs additionally move for summary judgment on their contributory 
copyright infringement claim. 8 Establishing proof of contributory infringement requires a party 
to demonstrate that the actor was “intentionally inducing or encouraging direct infringement.” 
MGMStudios, Inc. v. Grokster, Ltd., 545 U.S. 913, 930 (2005). Plaintiffs 9 must show (1) direct 
infringement by third parties; (2) that Defendant knew that third parties were directly infringing; 
and (3) that Defendant substantially participated in that direct infringement. Rundquist v. 

Vapiano SE, 798 F. Supp. 2d 102, 126 (D.D.C. 2011). “Merely supplying the means to 
accomplish an infringing activity cannot give rise to the imposition of liability for contributory 
copyright infringement.” Newborn v. Yahoo!, Inc., 391 F. Supp. 2d 181, 186 (D.D.C. 2005) 
(internal quotation omitted). 


8 The ASTM Plaintiffs initially brought a separate claim for contributory copyright 
infringement, but did not include that claim in their motion for summary judgment. Counsel for 
ASTM Plaintiffs stated at oral argument that they believed the remedy for their infringement 
claim covered any potential remedy for their contributory copyright claim. (Tr. of Motions 
Hearing at 122:1-7). 

9 Because ASTM Plaintiffs did not move for summary judgment on their contributory copyright 
claim, for this section the court will use “Plaintiffs” to refer to AERA Plaintiffs. 
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To establish direct infringement by third parties, Plaintiffs must demonstrate “(1) which 
specific original works form the subject of the copyright claim; (2) that the plaintiff owns the 
copyrights in those works; (3) that the copyrights have been registered in accordance with the 
statute; and (4) by what acts [and] during what time the defendant infringed the copyright.” Id. 
(quoting Home & Nature, Inc. v. Sherman Specialty Co., 322 F. Supp. 2d 260, 266 (E.D.N.Y. 
2004)). As discussed above in section III(A), these first three elements have been satisfied. On 
the fourth element, Plaintiffs must show that a third party infringed its copyrights by violating 
their exclusive rights under 17 U.S.C. § 106, including reproduction, preparation of derivative 
works, distribution, or public display. See Home & Nature, 322 F. Supp. 2d at 267. However, 
Plaintiffs only present evidence that the 1999 Standards were “accessed at least 4,164 times” on 
Public Resource’s website and that they were “accessed on the Internet Archive ... website 
1,290 times.” (AERA PSMF 85-86). Without more, there is no basis for the court to 
determine that accessing a website is equivalent to copying or violating any of the exclusive 
rights under § 106. Plaintiffs also assert that “some” individuals “obtained” the standards, but 
their only evidence of this is a redacted e-mail in which an individual states “[0]ne of my 
students showed up for class this semester and told me that he/she didn’t purchase a copy of the 
Standards (I require them as a text for one of my courses) because ‘they are available for free on 
line’ and they showed me the following site.” (Exl. LLL to Deck of Lauress Wise (AERA ECF 
No. 60-75)). Even if such a statement were ultimately determined to be admissible for the truth 
of the matter that the student did not purchase the Standards, it still does not establish that the 
student downloaded or otherwise copied the 1999 Standards from Defendant’s website. 10 


10 The court recognizes that acquiring evidence of downloads may be difficult. Carl Malamud, 
Public Resource’s CEO, testified at deposition that “I don’t know about downloads. It’s 
technically impossible to determine that.” (Ex. A to Hudis Decl. at 347:6-8 (AERA ECF No. 
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In their Reply Brief, Plaintiffs also point to the possibility that simply browsing a website 
causes a copy of the material on the website to be automatically copied to the computer’s random 
access memory or RAM. See CoStar Realty Info., Inc. v. Field, 737 F. Supp. 2d 496, 507 (D. 

Md. 2010) (analyzing copyright claim involving cache copies of websites in computer’s RAM); 
Ticketmaster, LLCv. RMG Techs., Inc., 507 F. Supp. 2d 1096, 1104-05 (C.D. Cal. 2007) (same). 
While this may be correct, the fact remains that Plaintiffs have put forth no actual evidence that 
even one of the 4,164 accesses resulted in such a copying to a computer’s RAM, and without 
such evidence, Plaintiffs cannot meet their burden on their contributory copyright claim at the 
summary judgment stage. 

The second two factors require Plaintiffs to establish that Defendant knew that third 
parties were engaged in direct infringement and that it substantially participated in such 
infringement. Plaintiffs may demonstrate knowledge by showing that Defendant was notified of 
the third party direct infringement or that it “willfully blind[ed] itself to such infringing uses.” 
Newborn, 391 F. Supp. 2d at 186. On this factor, Plaintiffs again fall short, relying on the fact 
that they asked Defendant to remove the 1999 Standards from its website and Defendant refused 
to do so, as well as evidence that Defendant did not track or prevent downloads of the 1999 
Standards from its website. Without more, this is insufficient to establish that Defendant knew 
that third parties were infringing the Plaintiffs’ copyrights. 

Similarly, Plaintiffs have not presented sufficient evidence on the substantial 
participation factor. While it is undisputed that Defendant posted the 1999 Standards on its 
website to enable greater access for those wishing to read them, because Plaintiffs have not 


60-4)). However, this does not relieve Plaintiffs of the burden of establishing some evidence 
demonstrating direct infringement by third parties. 
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established any actual third party direct infringement, there is insufficient evidence that 
Defendant substantially participated in that infringement. 

Therefore, the court DENIES Plaintiffs’ motion for summary judgment as to its 
contributory copyright claim, and also DENIES Defendant’s motion for summary judgment on 
this claim, as there exists questions of fact as to any third party infringement, Defendant’s 
knowledge, and Defendant’s participation. 

C. Trademark Infringement Claims 

ASTM Plaintiffs additionally moved for summary judgment on their trademark 
infringement, unfair competition and false designation of origin, and common law trademark 
infringement claims, and Defendant cross-moved for summary judgment on these claims as 
well. 11 Trademark law is governed by the Lanham Act, 15 U.S.C. § 1051 etseq., which provides 
that: 

(1) Any person who shah, without the consent of the registrant... (a) use in 
commerce any reproduction, counterfeit, copy, or colorable imitation of a 
registered mark in connection with the sale, offering for sale, distribution, or 
advertising of any goods or services on or in connection with which such use is 
likely to cause confusion, or to cause mistake, or to deceive . . . shah be liable in a 
civil action by the registrant for the remedies hereinafter provided. 

15 U.S.C. §1114(1). In order to prevail on a trademark infringement claim under the Lanham 

Act, Plaintiffs 12 “must show (1) that [they] own[] a valid trademark, (2) that [their] trademark is 

distinctive or has acquired a secondary meaning, and (3) that there is a substantial likelihood of 

confusion between the plaintiffs’] mark and the alleged infringer’s mark.” Globalaw Ltd. v. 

Carmon & Carmon Law Office, 452 F. Supp. 2d 1, 26 (D.D.C. 2006); AARP v. Sycle, 991 F. 


11 The AERA Plaintiffs did not bring a trademark claim, and so this section applies only to 
ASTM Plaintiffs. 

12 As in the preceding section, because only ASTM Plaintiffs moved for summary judgment on 
this claim, the court will refer to them here as Plaintiffs. 
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Supp. 2d 224, 229 (D.D.C. 2013) (same). Common law claims are analyzed under the same 
standard. See AARP, 991 F. Supp. 2d at 229 (citing Breaking the Chain Found., Inc. v. Capitol 
Educ. Support, Inc., 589 F.Supp.2d 25, 29 (D.D.C. 2008)). In order for conduct to be considered 
infringing, there must be a “use in commerce.” 15 U.S.C. §§1114(1), 1125(a)(1). 

Defendant cites Dastar Corp. v. Twentieth Century Fox Film Corp., to discourage the 
court from considering Plaintiffs’ trademark claims on the principle that courts should not 
“misuse or over-exten[d] [] trademark and related protections into areas traditionally occupied by 
patent or copyright.” 539 U.S. 23, 34 (2003). Dastar held that a plaintiff could not bring a false 
designation of origin trademark claim against a defendant who was distributing content that had 
become part of the public domain because the Lanham Act only offers protection “to the 
producer of the tangible goods that are offered for sale, and not to the author of any idea, 
concept, or communication embodied in those goods.” Id. at 37. Unlike in Dastar, Plaintiffs 
here have an independent basis for claiming that Defendant infringed their trademarks, separate 
from their copyright infringement claims: Defendant distributed standards online bearing 
Plaintiffs’ registered trademarks and logos, and Plaintiffs argue that this unauthorized use of their 
marks will confuse consumers and falsely signal that Plaintiffs are the origin of the standards 
distributed on Defendant’s website rather than Defendant. While the remedy sought for 
Plaintiffs’ copyright claim—an injunction barring Defendant from displaying Plaintiffs’ 
standards online—may be broad enough to subsume a remedy for their trademark claims, the 
claims are based on independent arguments, and are therefore the type that Dastar found to be 
appropriate for consideration under the Lanham Act. 

The court must therefore consider whether Plaintiffs own a valid, protectable trademark, 
whether Defendant engaged in an unauthorized use in commerce, whether there is a likelihood of 
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consumer confusion, and whether Defendant’s fair use defense permits its use of the trademarks. 

1. Valid, Protectable Trademark 

Under the Lanham Act, any registration of a trademark “shall be prima facie evidence of 
the validity of the registered mark and of the registration of the mark, of the owner’s ownership 
of the mark, and of the owner’s exclusive right to use the registered mark in commerce.” 15 
U.S.C. § 1057(b). The record indicates that Plaintiffs own valid trademarks of the trademarks 
asserted in this case, and they have federal trademark registrations for each of the asserted 
marks. 13 Thus, Plaintiffs have established a prima facie showing of ownership. Defendant 
offers no evidence to demonstrate that Plaintiffs do not own the trademarks, and therefore the 
court concludes that Plaintiffs are the owners of these marks. 

The trademarks must also be “valid.” To establish validity, Plaintiffs must prove that the 
designation is inherently distinctive or that it has become distinctive by acquiring secondary 
meaning. See Two Pesos, Inc. v. Taco Cabana, Inc., 505 U.S. 763, 769 (1992); Globalaw, 452 
F. Supp. 2d at 26. However, Plaintiffs’ trademark registrations create a rebuttable presumption 
of “inherent distinctiveness or secondary meaning.” Restatement (Third) of Unfair Competition 
§ 13 cmt. a (1995). Additionally, the Lanham Act provides that if the trademark has been “in 
continuous use for five years subsequent to registration” then the marks become “incontestable,” 

15 U.S.C. § 1065, meaning the registration “shall be conclusive evidence of the validity of the 
registered mark,” including as to whether it is distinctive or has a secondary meaning, 15 U.S.C. 

§ 1115(b); see also Restatement (Third) of Unfair Competition § 13 cmt. a (1995). Plaintiffs 

13 (PSMF 77 (trademark registration for “ASTM”), 78 (trademark registration for “ASTM 
International” and logo), 79 (trademark registration for ASTM logo), 123 (trademark registration 
for “National Fire Protection Association” and “NFPA”), 124 (trademark registration for NFPA 
logo), 126 (trademark registration for NEC logo), 149 (trademark registration for ASHRAE 
logo), 151 (trademark registration for additional ASHRAE logo)). 
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provided evidence that some of their trademarks have become incontestable and that they all are 
distinctive. {See PSMF 77, 78, 124, 125, 126, 150). Defendant offered no evidence to dispute 
the validity of the trademarks. Thus, Plaintiffs have sufficiently established their ownership of 
valid trademarks. 

2. Defendant’s Unauthorized Use in Commerce 

Plaintiffs must also demonstrate that Defendant used their trademarks “in commerce.” 15 
U.S.C. §§1114(1), 1125(a)(1). Under the Lanham Act, ‘“[cjommerce’ means all commerce 
which may be lawfully regulated by Congress.” 15 U.S.C. § 1127. Therefore, to satisfy this 
requirement, Plaintiffs need not demonstrate actual use or intended use in interstate commerce. 

See United We Stand Am., Inc. v. United We Stand, Am. N.Y., Inc., 128 F.3d 86, 92 (2d Cir. 

1997) (the commerce requirement “reflects Congress’s intent to legislate to the limits of its 
authority under the Commerce Clause, rather than to limit the Lanham Act to profit-seeking uses 
of a trademark”). Distribution on the Internet can satisfy the “use in commerce” requirement. 

See Intermatic, Inc. v. Toeppen, 947 F. Supp. 1227, 1239 (N.D. Ill. 1996). Thus, Defendant’s 
online posting of the standards bearing Plaintiffs’ trademarks satisfies this requirement. 

This use in commerce must further be “without the consent of the registrant.” 15 U.S.C. 

§ 1114(1). It is undisputed that Plaintiffs did not authorize Defendant’s use of Plaintiffs’ 
trademarks in commerce. Defendant instead argues that its use was permitted under the “first 
sale doctrine,” which holds that a trademark owner cannot control what happens to its products 
after the first sale. However, the court finds this doctrine a poor fit here, where it is undisputed 
that Defendant did not redistribute the physical copies of Plaintiffs’ standards that it purchased 
but rather created reproductions through scanning and re-typing, with resultant errors and 
differences. See Australian Gold, Inc. v. Hatfield, 436 F.3d 1228, 1241 (10th Cir. 2006) (noting 
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that the first sale doctrine is appropriate only when the actor “does no more than stock, display, 
and resell a producer’s product under the producer’s trademark”); Capitol Records, LLC v. 

DeRigi Inc., 934 F. Supp. 2d 640, 655 (S.D.N.Y. 2013) (in the copyright context, the first sale 
doctrine was “impossible” to apply because that defense is limited to when an actor distributes 
the original material item, not when she distributes reproductions). 

Moreover, Defendant’s quality control standards in reproducing Plaintiffs’ standards 
were outside of Plaintiffs’ control and below that sufficient to deem the standards it distributed 
“genuine” products, meaning the first sale doctrine cannot protect Defendant’s conduct. See 
Polymer Tech. Corp. v. Mimran, 37 F.3d 74, 78 (2d Cir. 1994); Shell Oil Co. v. Commercial 
Petroleum, Inc., 928 F.2d 104, 107 (4th Cir. 1991); El Greco Leather Prods. Co. v. Shoe World, 

806 F.2d 392, 395 (2d Cir. 1986); see also 4 McCarthy on Trademarks and Unfair Competition 
§ 25.42 (4th ed.). Although Defendant argues that there are no material differences between 
Plaintiffs’ standards and Defendant’s reproductions, Plaintiffs need not show that Defendant’s 
reproduced standards were defective, only that they were unable to exercise quality control. See 
Zino DavidoffSA v. CVS Corp., 571 F.3d 238, 243 (2d Cir. 2009). The claim survives because 
“the interference with the trademark holder’s legitimate steps to control quality unreasonably 
subjects the trademark holder to the risk of injury to the reputation of its mark.” Id. Plaintiffs 
have established that Defendant’s quality control standards, including “double-keying” the 
standards, a process involving two separate individuals typing the same material and comparing 
the results to determine the existence of any errors, resulted in missing or inverted pages and 
typographical errors in numerical values or formulas. (ASTM PSMF 190, 214-15). Because 
the standards are therefore not “genuine,” the first sale doctrine does not apply, and Plaintiffs 
have established that Defendant used its trademarks in commerce without authorization. 
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3. Likelihood of Confusion 

Next, the court must assess whether there is a substantial likelihood of consumer 
confusion. This hinges on whether “an appreciable number of ordinarily prudent customers are 
likely to be misled, or simply confused, as to the source” of the copied standards that Public 
Resource posted online. Globalaw, 452 F. Supp. 2d at 47. 

Plaintiffs argue that consumers will be confused both in thinking that Plaintiffs 
authorized Defendant’s posting of the standards, and that Plaintiffs produced the PDF and 
HTML versions of the standards that Defendant posted. See Am Ass ’n for the Advancement of 
Science v. Hearst Corp., 498 F. Supp. 244, 258 (D.D.C. 1980) (noting that both are appropriate 
bases for a confusion argument). Courts in this Circuit consider approximately seven factors in 
assessing the likelihood of confusion, though none is individually determinative. Globalaw, 452 
F. Supp. 2d at 48. They include: (1) the strength of the Plaintiffs’ marks; (2) the degree of 
similarity between the marks; (3) the proximity of the products; (4) evidence of actual confusion; 
(5) Defendant’s purpose or reciprocal good faith in adopting its own mark; (6) the quality of 
Defendant’s product; and (7) the sophistication of the buyers. Id. Several courts in other 
Circuits have determined that when a defendant uses an identical mark on a similar product, 
consideration of all the factors is not necessary. Seelnt’l Cosmetics Exch., Inc. v. Gapardis 
Health & Beauty, Inc., 303 F.3d 1242, 1248-49 (11th Cir. 2002); Wynn Oil Co. v. Thomas, 839 
F.2d 1183, 1190-91 (6th Cir. 1988). 

Defendant does not dispute that Plaintiffs’ marks are “strong,” that Defendant used marks 
and logos that are identical to Plaintiffs’ marks and logos when it posted the Plaintiffs’ standards 
online, and that the standards it applied the marks and logos to were identical or nearly identical 
to Plaintiffs’. (PSMF fflf 210-11; Def. Br. at 65). Moreover, it is undisputed that the standards 
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distributed by Plaintiffs and by Defendant were in close proximity, since Defendant offered the 
standards in the same market as Plaintiff—i.e., the Internet—as a free alternative to purchasing 
the standards from Plaintiffs directly. See Restatement (Third) of Unfair Competition § 21 cmt. j 
(1995) (“[T]he use of similar designations on goods that are used together, or that perform the 
same function, or that are of the same general class, is more likely to cause confusion than is a 
use in connection with goods used for different purposes, or in different contexts, or by different 
purchasers.”). It is also undisputed that Defendant intended for individuals to consider that the 
standards were identical. (PSMF1213). 

Defendant argues that despite these undisputed facts, consumers would not be confused 

because it posts disclaimers that it claims “adequately informed consumers” so that “no 

reasonable consumer would mistake [its cover page] as part of the original document.” (Def. 

Reply at 28 (referring to the PDF disclaimer at ASTM ECF No. 118-12, Ex. 16)). Defendant 

also argues that the PDF versions it posted “look like scans of physical documents,” and that the 

“preamble for the .html standards informs reasonable consumers that Public Resource has 

provided the transcription.” {Id. (referring to the HTML disclaimer at ASTM ECF No. 118-13, 

Ex. 26)). 14 Here, Defendant’s disclaimer on the PDF reads in full: 

In order to promote public education and public safety, equal justice for all, a 
better informed citizenry, the rule of law, world trade and world peace, this legal 
document is hereby made available on a noncommercial basis, as it is the right of 
all humans to know and speak the laws that govern them. 

(ASTM ECF No. 118-12, Ex. 16). The disclaimer on the HTML versions contain s similar 


14 Defendant cites to Prestonettes, Inc. v. Coty, 264 U.S. 359, 369 (1924), in support of its 
argument that a disclaimer is sufficient to inform consumers that it has repackaged or changed 
the original. The facts of that case do not support Defendant’s position, as the disclaimer in that 
case stated clearly that the distributor was not connected with the producer and that the 
producer’s product was merely a constituent part of the distributor’s new product. Coty, 264 
U.S. at 367. 
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language. (ASTMECFNo. 118-13, Ex. 26). These disclaimers do not mention Defendant’s 
creation of the reproductions, Plaintiffs’ lack of association or authorization, or that they are even 
reproductions or transcriptions, and can hardly be called disclaimers at all. Moreover, 

Defendant’s assertion that the PDFs “look like scans” offers no assistance to a consumer looking 
at the standard, as they would have no way to determine whether the Plaintiffs or Defendant 
created the scan. While Defendant has since adopted a more thorough disclaimer that includes 
information about Public Resource’s retyping of the HTML versions and the possibility of errors 
(DSMF ]f 169), it did not begin using that disclaimer until 2015, after the start of this litigation. 
(Decl. of Carl Malamud 131 (ASTM ECF No. 122-8)). 

The parties have presented no evidence to establish the existence or non-existence of 
actual consumer confusion. While such evidence is not required, without it summary judgment 
on consumer confusion, and trademark infringement more generally, is a difficult call. However, 
the facts here present nearly as black-and-white a case as possible. A consumer in the market for 
one of Plaintiffs’ voluntary consensus standards may encounter them on Plaintiffs’ websites for 
purchase, or on Defendant’s website for free download. Because Defendant has intentionally 
created a copy that is meant to appear identical, including use of Plaintiffs’ trademarks, then that 
consumer may download that standard for free from Defendant without knowing that it is not 
created by the Plaintiffs and may contain missing pages or typographical errors leading to 
inaccurate values for measurements. In short, Plaintiffs have presented enough evidence for the 
court to conclude that there is no genuine dispute on the factual issue of whether consumer 
confusion is likely. 

4. Defendant’s Nominative Fair Use Defense 

While Plaintiffs have successfully established Defendant’s infringing use of their 
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trademarks, Defendant argues that its use of Plaintiffs’ trademarks is “nominative fair use.” 

Under this defense, Defendant must demonstrate that its use of Plaintiffs’ trademarks was 
necessary to describe their standards; that it only used as much of the marks as was reasonably 
necessary to identify the standards; and that it has not done anything to suggest sponsorship or 
endorsement by the Plaintiffs or to inaccurately describe the relationship between the parties’ 
products. See Rosetta Stone Ltd. v. Google, Inc., 676 F.3d 144, 154 (4th Cir. 2012). Nominative 
fair use by a defendant makes it “clear to consumers that the plaintiff, not the defendant, is the 
source of the trademarked product or service.” Century 21 Real Estate Corp. v. Lendingtree, 

Inc., 425 F.3d 211, 220 (3d Cir. 2005). Thus, if Defendant’s use is nominative fair use, it would 
not create “confusion about the source of [the] defendant’s product.” Tiffany (NJ) Inc. v. eBay 
Inc., 600 F.3d 93, 102 (2d Cir. 2010) (alteration in original). On this point, the parties argue past 
each other. Defendant believes no consumer would believe that Defendant, rather than Plaintiffs, 
was the source of the standards, and so its use is a fair use. Plaintiffs argue that Defendant’s use 
ca nn ot be fair precisely because consumers would believe that Plaintiffs were the source of the 
reproduced standards, which they are not. However, because the court has already determined 
that consumer confusion as to the source of the trademarked standards is likely, the nominative 
fair use defense is inapplicable and the court need not assess each of the Rosetta Stone factors 
listed above. 

The court therefore finds that Defendant engaged in trademark infringement by its use of 
Plaintiffs’ registered trademarks, and Plaintiffs’ motion for summary judgment on their 
trademark claims is GRANTED and Defendant’s cross-motion is DENIED. 

IV. REMEDIES 

Both ASTM Plaintiffs and AERA Plaintiffs seek a permanent injunction barring 
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Defendant from distributing, displaying, or creating derivative works from their copyrighted 
standards and, in the case of ASTM Plaintiffs, their trademarks, which this court has authority to 
grant under 17 U.S.C. § 502(a) (Copyright Act) and 15 U.S.C. § 1116 (Lanham Act). Plaintiffs 
must establish (1) irreparable injury; (2) that remedies available at law, such as monetary 
damages, are inadequate to compensate for their injury; (3) that a remedy in equity is warranted 
after considering the balance of hardships; and (4) that the public interest would not be disserved 
by a permanent injunction. See eBay Inc. v. MercExchange, LLC, 547 U.S. 388, 391 (2006). 

A. Irreparable Injury 

The ASTM Plaintiffs assert that they will face three separate irreparable injuries if 
Defendant is permitted to continue distribution of Plaintiffs’ standards, including substantial 
declines in revenue that may cause their business models to change, the loss of the exclusive 
rights under the Copyright Act to exclude others from distributing, reproducing, or displaying 
their protected works, and the loss of control of the goodwill associated with their trademarks. 
AERA Plaintiffs similarly assert that they will face three separate irreparable injuries if 
Defendant is permitted to continue distribution of Plaintiffs’ standards, including loss of business 
opportunities, the loss of the exclusive rights under the Copyright Act to exclude others from 
distributing, reproducing, or displaying their protected works, and the adverse effect on 
Plaintiffs’ efforts to create further standards. 

It is well established that the threat of continuing copyright infringement justifies 
granting a permanent injunction. See Walt Disney Co. v. Powell, 897 F.2d 565, 567 (D.C. Cir. 
1990) (“When a [ ] plaintiff has established a threat of continuing infringement, he is entitled to 
an injunction.”); Hanley-Wood LLC v. Hanley Wood LLC, 783 F. Supp. 2d 147, 151 (D.D.C. 

2011); Breaking the Chain Found, v. Capital Educ. Support, Inc., 589 F. Supp. 2d 25, 30 
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(D.D.C. 2008). While a court should not automatically issue an injunction after it finds there 
was past copyright or trademark infringement, here Plaintiffs’ alleged irreparable injury is not 
the past infringement but the threat of future infringement. Defendant has not provided any 
assurances that it would cease posting of Plaintiffs’ standards—indeed, it is undisputed that 
during the course of this litigation, Public Resource posted online versions of the ASTM 
Plaintiffs’ other standards not involved in this litigation. (PSMF 235). Moreover, Defendant’s 
counsel at oral argument admitted that Defendant would post the AERA Plaintiffs’ 2014 
Standards if they were incorporated by reference into federal regulations in the future. (Tr. of 
Motions Hearing at 75:24—76:2). The court thus determines that the continued threat of 
infringement is sufficient to weigh in favor of an injunction. 

B. Adequacy of Monetary Damages 

Plaintiffs argue that because damages here are difficult to quantify and Defendant may be 
unable to pay damages, then legal remedies are inadequate. See Fox Television Stations, Inc. v. 
FilmOn XLLC, 966 F. Supp. 2d. 30, 50 (D.D.C. 2013). The evidence shows that while the 
Plaintiffs’ standards were accessed thousands of times on Defendant’s website, Defendant does 
not track information that would be helpful in calculating damages, such as how many of those 
accesses actually led to downloads, and whether those downloads were in lieu of purchases. 
Moreover, Defendant did not dispute that it has “extremely limited financial resources available 
to pay any damages award” and that in 2014 it “generated under $100,000 in operating income 
and had $248,000 in total net assets.” (ASTM PSMF fflf 272-73). Given that the Copyright Act 
provides for statutory damages ranging from $750 to $30,000 for each of the standards at issue in 
the overall case, or even up to $150,000 per infringement if Plaintiffs were to later prove that 
infringement was committed willfully, Defendant’s potential inability to pay is surely a factor 
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weighing towards equitable relief. See 17 U.S.C. § 504(c)(l)-(2). 

C. Balance of Hardships & Public Interest 

The court must weigh the likely harms faced by Plaintiffs described above with any 
harms faced by Defendant if an injunction is imposed. Here, Defendant’s CEO Carl Malamud 
was asked in his ASTM deposition what financial impact an injunction barring posting of the 
standards would have on Public Resource, and he responded “probably none.” (Malamud Dep. 
at 219:22-220:4 (Ex. 3 to Rubel Deck (ASTM ECF No. 118-12))). The only harm Mr. Malamud 
identified was that “one hates to have wasted that [] effort” that went into posting the standards 
online. (Id.). Without evidence of any additional harms, this factor weighs strongly in favor of 
an injunction. 

Additionally, the public must not be disserved by the issuance of an injunction. Here, the 
public interest is served by the policy interests that underlie the Copyright Act itself, namely the 
protection of financial incentives for the continued creation of valuable works, and the continued 
value in maintaining the public-private system in place in the U.S. to ensure continued 
development of technical standards. 

Taken together, the court finds that injunctive relief is appropriate and that Defendant 
should be permanently barred from violating any of Plaintiffs’ exclusive copyrights, including 
distributing, displaying, reproducing, or creating derivative works in the nine standards on which 
ASTM Plaintiffs moved for summary judgment and AERA Plaintiffs’ 1999 Standards, as well as 
barred from any use of ASTM Plaintiffs’ trademarks in connection with the posting of these 
standards online or elsewhere. 

V. CONCLUSION 

For the reasons set forth above, ASTM Plaintiffs’ Motion is GRANTED, AERA 
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Plaintiffs’ Motion is GRANTED IN PART and DENIED IN PART, and Defendant’s Cross- 
Motions are DENIED. 


Date: February 2, 2017 






TANYA S. CHUTKAN 


United States District Judge 
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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


) 

) 

) 

) 

) 

) 

) Case No. 14-cv-0857 (TSC) 

) 

) 

) 

) 

) 

ORDER 

Upon consideration of the parties’ motions, and for the reasons set forth in the court’s 
Memorandum Opinion, Plaintiffs’ motion is GRANTED IN PART and DENIED IN PART and 
Defendant’s motion is DENIED. 

It is hereby ORDERED that Defendant is permanently enjoined from all unauthorized 
use, including through reproduction, display, distribution, or creation of derivative works, of the 
Standards for Educational and Psychological Testing, 1999 edition. 

Defendant is FURTHER ORDERED to remove all versions of this standard from its 
website and any other website within its possession, custody, or control within five days. 


AMERICAN EDUCATIONAL 
RESEARCH ASSOCIATION, INC. et al.. 

Plaintiffs, 

v. 

PUBLIC.RESOURCE.ORG, INC., 
Defendant. 


Date: February 2, 2017 




TANYA S. CHUTKAN 


United States District Judge 


JA3456 






Case l:14-cv-00857-TSC Document 119 Filed 02/10/17 Page 1 of 4 
USCA Case #17-7035 Document #1715850 Filed: 01/31/2018 Page 435 of 441 


UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN EDUCATIONAL RESEARCH ) 

ASSOCIATION, INC., AMERICAN ) Civil Action No. 1:14-cv-00857-TSC-DAR 

PSYCHOLOGICAL ASSOCIATION, INC., ) 

and NATIONAL COUNCIL ON ) PLAINTIFFS’ MOTION FOR 

MEASUREMENT IN EDUCATION, INC., ) CLARIFICATION OF THE COURT’S 

) ORDER DATED FEBRUARY 2, 2017 
Plaintiffs/Counterclaim-Defendants, ) AND, IN THE ALTERNATIVE, 

) CONSENT MOTION FOR 

v. ) CONTINUANCE OF DEADLINES 

) FOR A MOTION FOR ATTORNEYS’ 
PUBLIC.RESOURCE.ORG, INC., ) FEES AND BILL OF 

) COSTS, WITH STATEMENT OF 
Defendant/Counterclaim-Plaintiff. ) POINTS AND AUTHORITIES 

_ ) 

Plaintiffs, American Educational Research Association, Inc. (“AERA”), American 

Psychological Association, Inc. (“APA”) and National Council on Measurement in Education, 

Inc. (“NCME”) (collectively, “Plaintiffs”) move herein for Clarification of the Court’s Order 

Dated February 2, 2017 (the “Order” - Dkt. No. 118) and, in the alternative, for Continuance of 

the Deadlines for File a Motion for Attorney’s Fees and a Bill of Costs. Counsel for the parties 

have met and conferred pursuant to Local Rule 7(m), and counsel for Defendant has consented to 

the enlargement of time as requested herein. 

STATEMENT OF POINTS AND AUTHORITIES 

Plaintiffs seek clarification as to whether the Order triggered the deadlines under Federal 

Rule of Civil Procedure 54(d) for filing motions for fees and costs, and, if so, to request a 

continuance of the deadlines until the matter has been fully and finally resolved and a final 

judgment has been entered by the Court. The potential deadlines for the motion for fees and 

costs are February 16 and February 23, respectively. In support of this Motion, Plaintiffs state as 

follows: 
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1. On February 2, 2017, this Court entered the Order granting in part and denying in 

part Plaintiffs’ Motion for Summary Judgment. Specifically, the Court directed that: 

It is hereby ORDERED that Defendant is permanently enjoined from 
all unauthorized use, including through reproduction, display, 
distribution, or creation of derivative works, of the Standards for 
Educational and Psychological Testing, 1999 edition. 

Defendant is FURTHER ORDERED to remove all versions of this 
standard from its website and any other website within its possession, 
custody, or control within five days. [Dkt. 118] 

2. The Copyright Act permits a party to seek recovery of costs and reasonable 
attorneys’ fees if it prevails in an action under the Copyright Act. See 17 U.S.C. §505 (“[T]he 
court in its discretion may allow the recovery of full costs by or against any party other than the 
United States or an officer thereof. Except as otherwise provided by this title, the court may also 
award a reasonable attorney’s fee to the prevailing party as part of the costs.”). 

3. Because Plaintiffs prevailed on summary judgment on Count 1, their claim of 
direct copyright infringement, and obtained a permanent injunction against Defendant for 
unauthorized use of Plaintiffs’ copyrighted work, Plaintiffs would be permitted, pursuant to the 
Copyright Act, to seek their reasonable attorneys’ fees and costs as to Count 1. 

4. However, the Order only granted in part Plaintiffs’ Motion for Summary 
Judgment and did not dispose of all claims in this action, leaving open Count 2 (contributory 
infringement) and Defendant’s Counterclaim. See Fed. R. Civ. P. 54(b) (“When an action 
presents more than one claim for relief. . . , the court may direct entry of a final judgment as to 
one or more, but fewer than all, claims . . . only if the court expressly determines that there is no 
just reason for delay. Otherwise any order or other decision, however designated, that 
adjudicates fewer than all claims . . . . does not end the litigation as to any of the claims . , , 
and may be revised at any time before the entry of a judgment adjudicating all claims . . . .”). 
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(emphasis added). 

5. Rule 54(b) counsels that filing a motion for attorneys’ fees and bill of costs on an 
order partially granting a motion for summary judgment is premature. However, a “judgment” is 
defined in Fed. R. Civ. P. 54(a) as “a decree and any order from which an appeal lies” 
(emphasis added). Pursuant to 28 U.S.C. § 1292(a)(1), an interlocutory order may be appealed 
from where it involves the “granting, continuing, modifying, refusing or dissolving injunctions, 
or refusing to dissolve or modify injunctions.” See 28 U.S.C. § 1292(a)(1) (“[T]he courts of 
appeals shall have jurisdiction of appeals from . . . interlocutory orders of the district courts of 
the United States ... or of the judges thereof, granting, continuing, modifying, refusing or 
dissolving injunctions, or refusing to dissolve or modify injunctions . . . .”). 

6. In granting an injunction, the Order in this case could under one lens be viewed to 
meet the definitional requirement of a “judgment” under Fed. R. Civ. P. 54(a), triggering the 
various deadlines outlined in Fed. R. Civ. P. 54(d) to file a motion for attorneys’ fees and bill of 
costs. 

7. However, because the Order decided fewer than all claims and did not constitute a 
final judgment, thereby making Rule 54(d) relief premature until a final judgment is entered by 
the Court, Plaintiffs request confirmation from the Court that the Order is not a “judgment” that 
triggers the Rule 54(d) deadlines. Otherwise, Plaintiffs’ motion for fees and bill of costs 
encourage piecemeal litigation, given that claims in the case remain outstanding, and would 
require the breaking out of time and costs spend on a single count within the Complaint. 

8. Should the Court determine that the Order is a “judgment” as defined under Fed. 

R. Civ. P. 54(a) upon which Plaintiffs may file a motion for attorneys’ fees and bill of costs 
pursuant to Fed. R. Civ. P. 54(d), Plaintiffs request that the deadlines pertaining to fees and costs 
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be continued until the matter has been fully and finally resolved and a final judgment has been 
entered by the Court. 1 

WHEREFORE, Plaintiffs American Educational Research Association, Inc., American 
Psychological Association, Inc., and National Council on Measurement in Education, Inc. 
respectfully request clarification from the Court confirming that the Order is not a judgment 
upon which Plaintiffs may seek attorneys’ fees and costs pursuant to Federal Rule of Civil 
Procedure 54(d). If the Court views the Order as a judgment upon which Plaintiffs may seek 
attorneys’ fees and costs pursuant to Federal Rule of Civil Procedure 54(d), Plaintiffs 
respectfully request that the deadlines to file any motion for attorneys’ fees and bill of costs be 
continued until the matter has been fully and finally resolved and a final judgment has been 
entered by the Court, and to grant such other relief that this Court deems just and appropriate. 

Respectfully submitted, 

QUARLES & BRADY LLP 

Dated: February 10, 2017 By: /s/ Jonathan Hudis _ 

Jonathan Hudis (DC Bar # 418872) 

Nikia L. Gray (Pro Hac Vice) 

Jonathan P. Labukas (DC Bar # 998662) 

1700 K Street NW, Suite 825 

Washington, DC 20006-3825 

Tel. (202) 372-9600 

Fax (202) 372-9599 

E-Mail JonathanUudis@quarles.com 

E-Mail Nikia.Gray@quarles.com 

E-Mail Jonathan.Labukas@quarles.com 

Counsel for Plaintiffs American Educational 
Research Association, Inc., American 
Psychological Association, Inc., and National 
Council on Measurement in Education, Inc. 


1 Defendant was cited in an article published by Law360 dated February 3, 2017, attached hereto as Exhibit A, as 
having “said [that Public.Resource.Org] planned to appeal the ruling” dated February 2, 2017. See Exhibit A, 11. 21- 
23. 
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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN EDUCATIONAL RESEARCH 
ASSOCIATION, INC.; 

AMERICAN PSYCHOLOGICAL ASSOCIATION, 
INC.; and 


Case No. l:14-CV-00857-TSC-DAR 

NOTICE OF APPEAL BY 
DEFENDANT-COUNTERCLAIMANT 
PUBLIC.RESOURCE.ORG, INC. 


NATIONAL COUNCIL ON MEASUREMENT IN 
EDUCATION, INC., 


Action Filed: May 23, 2014 


Plaintiffs/Counter-Defendants, 


PUBLIC.RESOURCE.ORG, INC. 


Defendant/Counterclaimant. 


Pursuant to 28 U.S.C. § 1292(a)(1), Defendant and Counterclaimant 
Public.Resource.Org, Inc. hereby gives notice of its appeal to the United States Court of Appeals 
for the District of Columbia Circuit from the order of February 2, 2017, Dkt. no. 118, 
permanently enjoining Public.Resource.Org and granting in part Plaintiffs’ Motion for Summary 
Judgment and Entry of a Permanent Injunction, pursuant to this Court’s decision in the 
memorandum opinion of the same date, Dkt. no. 117. 
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Dated: February 17, 2017 


Respectfully submitted, 


/s/ Andrew P. Bridges _ 

Andrew P. Bridges (admitted) 

abridges@fenwick.com 

Sebastian E. Kaplan (admitted pro hac vice ) 

skaplan@fenwick.com 

Matthew Becker (admitted pro hac vice ) 

mbecker@fenwick. com 

FENWICK & WEST LLP 

555 California Street, 12th Floor 

San Francisco, CA 94104 

Telephone: (415) 875-2300 

Facsimile: (415)281-1350 

Corynne McSherry (admitted pro hac vice ) 
cor 5 mne@eff. 0 rg 

Mitchell L. Stoltz (D.C. Bar No. 978149) 
mitch@eff.org 

ELECTRONIC FRONTIER FOUNDATION 

815 Eddy Street 

San Francisco, CA 94109 

Telephone: (415)436-9333 

Facsimile: (415)436-9993 

David Halperin (D.C. Bar No. 426078) 
davidhalperindc@gmail.com 
1530 P Street NW 
Washington, DC 20005 
Telephone: (202) 905-3434 

Attorneys for Defendant-Counterclaimant 
Public.Resource.Org, Inc. 
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CERTIFICATE OF SERVICE 


I hereby certify that the foregoing was electronically filed with the Clerk of the Court for 
the District of Columbia and served on all counsel of record via the CM/ECF system on 
February 17, 2017. 


/s/ Andrew P. Bridges 

Andrew P. Bridges 
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